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ABSTRACT 

Human factors engineering can be defined as the 
application of scientific principles, methods, and data drawn f rbip. a 
variety of disciplines to the development of engineer ihg systems in 
v^hich people play a significant role . Since human factor s issues 
arise in every domain in which humans interact with the products of a 
tecfiholbgical society, six areas ih_ which basiv^ research is needed to 
improve the sciehtif ic basis of applied human factors work have been 
identified. These areas include :( 1) human decision-making; (2) 
eliciting information from experts; (3) user-computer interaction; 
(4l supervisory control systems; (5) population group^dif f erences; 
and (6) applied methods (including education in this field) . Each 
area is discussed in a separate chapter designed to be a 
self-contained report of the research needs in that area. (JN) 
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The eon„nltt.e on Hu^n Factors ^as established in Oet.b.r 1980 under 
the joint .^^ons.rshlp of the office .f Naval Research (ONR). the Air 
• Force Office of Scientific Research (AFOSR). and the A?my Ra...rch 
Institute for the Behavioral aod Social Sciences (ARI) to identify 
b..ic research "needs of ^ fhe military service. In support of huo^n ' 
- factors engineering Applications and to reco,:„.ektions for bSsic 
research^iha^ Will i:,prove the. foundations of this jiisclpilne. The 
•co«m,it tee's first .eating J,eld -in December^ 1980; In October 1981 
the National Aeronautics and Sp«e Administration (NASA) joined the 
sponsors Of the co^lttee; .nd i.veral other goverr^.nt agencies have 
expressed interest "in the cpmmittee 's work. 

Human factors issues aris. in every domain in which humans 
interact with the proHucts of a technological society. Consequently, 
the knowledge brought to bear In hum.n factors appiic.tix,ns must be 
d^a^from a widi range scientific .nd engineering discip'llnes . ^ 
Although no s.all group can be fully -rep-^sent^tlve of .11 disciplines 
relevant to hum.n factors, the' expertise represented on th. committee 
is quite brc4d. It, incltldes specialists from th^ fields of " " ' 
engineering, biomechanics, psychology, cognitive science, and . 
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' .i n fro,;, hb.i,an factors engineering; While other 

^'.sctpim.s tnay be relevant, it Is these that «e expected to 
contribute .o^ iub.tantially to tlie b..ic data, theory, a.d ^nithods 
heeded. to improve t;ie scientific basis of humah factors. \ 
^ I wish to thank .ach member of ihe committee for their thdughtful 

- contributions to thii report. Individual ^.embers or email groupS^of 
members accepted p.ln^ry responsibility for authoring each chapter. 
This authorship Is acknowledged In the note .t the beginning of each 
Chapter. .All committee members. Whether they were authorS or not. 
deliberated, reviewed, and contributed to Improvements in the cont..^ 
Of .ach chapter. I am- especially grateful to them for their generous - 
contribution, of time, both In meetings and outside. Their efforts 
have C0htrlbute*4reatly to tfre quality of this report, which U 
a product Of the full committee. Special thanks ire due to the itudy 
director. Robert T^ehnessy. who contributed both technically and 
administratively to every step In the report's development. In • 
addition. -he has. taken the kind of < Initiatives that made It possible 
for me to chair the committee with minimum effort and'maxlmum reward. f 

Martin A. TolcOtt and Gerald S. Maleckl of the Office of Naval 
Research. Alfred R. Fregly of the Air Force Offi^ of Scientific 
Research. Robert M. Sasmor of the Army Research Institati. and Meivlxi " 
D. Montemerlo of. the National Aeronautics and Space Administration, 
representatives Of <he committee's sponsors, have alsomade Important 
contributions. Their support, encouragement, and Identification of 
relevant iisues have been most helpful. * ' . - ~ 
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I am grateful also to tHe participantb in our workshop on applied 
methods: Stuart K. Card, David Melst^r, Donald L; Parke, Erich P. 
Prleh, and John B. Shafer. , Their broa<| Uhderstaridlrig of ipplied 
methods and their cogent appraisal of the Issues and needs in this 
area formed the basis for Chapter 7 of this report. 

Several people were helpful to the committee In specific ways. 
At Wrlght-Palterson Air Force Base, Kenneth M. Bdff organized a serleb ' 
of briefings by . personnel from the Air Force Aerospace Medical- 
^ Research Labbratbr^^^^nd the Air Force Humar^esoarces Labbratbry as 
well as tours bf several of their research facilities. During the 
committee's visit tb^the«Naval Training Equipment Center, Walter S. 
Chambers and Stanley C' Cbllyer arranged for presentations by members^ 
of the Human Factors. Labbratbry arid briefed the committee on the 
research uses of the visual technology research simulator as well as 
demonstrating this device. I extend my appreciation to these 
Individuals and organizations for their efforts on the committee *6 ' 
behalf. ^ - 

^ Many other individuals also have cbritributeii to the^ork of the 

cbnSnlttee and thereby to the contents bf this ref)brt. A number of 
human factors professionals provided thoughtful arid detailed"1^esp^ses 
to a survey bri research issues.' Others served as outside reviewers bf 

• . particular chapters. Karen A. English and M. Jearirie Richards have 

\ ' ^ _ - 

served ablyCand conscientiously as administrative secretaries over the 
- - - - ■ . 

course bf the committee *s history, ehristine L. McSharie^ editor for 

the Commission on Behavioral and Social Sciences atsd Education^ 

through skill arid p«rseverence greatly improved the style arid clarity 
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tin rt-,;rST-..- ail th^^se individuals 1 express itSc«e thanks 
for Eheir significant cbhtrlbutlbnsi 

. The committee's work i«j1*igolng. This Is the first In Whit is 
expected to be m c«5ntlnulng series of reports on issues In hon«S 
factors research. I Invite the reader's comments and reactions- tb- 
. this ahi fUtare reports , • 'v: ^ ' 



Richard Hf ?ev. Chair 
. Committee on Human Factors 
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JJiTRODUCTION" AND OVERViEM 



m the last several years the public has become sensitized to the 
Importance of equipment designed to accommodate Its' finman' users > In 
the course of events at the Thi-e; Mile Island nuclear pow^ plant many 
resldentys of Harrlsburg were evacuated because of the accident ^ ' 
precipitated by operators misinterpreting their Instruments. Coal 
miners cover equipment lamps intended to illuminate the' mine waii, 
, because they object , to the glare in th^ir faces. The H-1, the most ; 
technoioglcaily sophisticated battle tink ever produced, is limited by 
the operating difficulties experienced by its cre«. With computer . - 
terminals now pervasive In thj workplace, more users are voicing their 
complaints about requirements to converse In arcane dlalect.6 of 

- ! ■ ' I * 

computer- languages . . V 
\ : 
Each of these examples reflects a failure to c5Hiider thfe design. 

of a system from the point of view of Its potential users; Fhus It is 
not surprising that the pobitc Is demanding that more attention be 
paid to such considerations. ITies^ demkr^ may Be expressed in the 
decisions of durors in court cases irivblving product liability. In the 
renewed- emphasis on^ hu-ian factors In military and ^aeronautics 



The principal author 6f this chapter Is Richard W. Pew* 
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labbratbrlee^ and In the Increase In job bppbrtutiltles ,fbr human 
factors professionals In the computer Industry. In Hatch 1982, over 
lydob people participated In a conference devoted to discussing hoi^ to 
make computers more user^oxlented. , 

The historical roots of the human factors profession are In 
industrial engineering and in psychol^Dgy. In the early l^OOs 
Frederick W. Taylor coined the term sclentl ^l^ m a n a gement , by which he 
meant the application of scientific principles in the design of the 
industrial workplace, Although overzeaious -Tayiorism** re^sulted 
some early mismanagement, hisvwork formed one of the building blocks 
for moderni Indus trial engineering and operations research. 

e "■ _ ^ 

During the latter stages of World War 11^ psychologists, wrc 

. - ♦ - 

been involved ir^ the selection and training of aircraft pilots, were 

,1 " ' . _ _ _ ^" _ ^. 

cabled on to take a novel perspective. Instead bf selecting, pilots to 

meet the severe, demands bf the cbckpit^ they were asked to select the 

cbcfepit design best suljted to the characteristics of pilots. This 

approach reduced accidents arid allowed a larger population of 

pbteritial pilbts tb be certified. Because flying pushes the human 

bbdy tb Its physiblbgical limits , the effects of physiological stress 

on perfbrmarice became a further consideration. ^After the war a small - 

^ ' ■ . Q_ _ ^ - _ 

grbup bf uriiversities b^gari training human factors Specialista for 
research arid develbpmerit In the military services arid the aerospace 
iridus{ty^ ^ ■ 

In 1957 the Human Actors Society was formed with 90 founding 
members; 1977 the metnbership had gfown to 1,956; and in tfie last 
five years the organization has expanded by an additional 50 percent, 
in addltlotti various engineering societies have formed groups related 
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to human factors; The formation of this committee within the National 
Research Council in 1980 is the latest explicit recognition of the 
importance of human factors in today 'a technological society* 

Human factors eligineering can be defined aa the application of 
scientific principles, iethods^ and data drawn from a variety of 
dis^ciplines to the development of engineering systems in which people 
play a significant role. Successful application is measured by 
Improved productivity^ efficiency, safety^ and acceptance of the 
resultai^t system design* The disciplines that may be applied to a 
particular problem include psycholbgyi cognitive sclericei physiology^ 
biomechanics^ applied physical anthropology ^ and Ini^ustrlal and 
systems engineering. The systems range from the use of a simple tool 
^^'"*'By-fi^onBUffler to multiperson sociotechnlcal systems. They typically 
include both technological and human components. 

Human factory specia/lists froir these and other disciplines are 
united by a singular perspective on the system design process: that 
desigtt begins with an understanding of the user^s role in overall 
system performance and that systems exist to serve their users, 
whether they are consumers^ system operators^ production workers^ or 
maintenance crews. This user-oriented design philosophy acknowledges 
human variability as a design parameter. The resultant designs 
incorporate features that take advantage of unique human capabilities 
as well as build in safeguards to avoid or reduce the Impact of 
unpredictable human error. 

On the Ihterhatlohal icenc this collection of activities has been 
called ergonomics^ meaning the study of work. . Its practitioners have 
placed somewhat more emphasis oh biomechanics and the physiological 

in ' 



coRts of doing voric than have human factors practltloiiers lis the 
United States* Aside from this distinction^ the two terias refer te 
the same vcdllectibh of specialties* 

While its fbuhdatibhs rest ultimately in the parent disciplines, 
human factors research focuses on the solution of system design 
problems involving more than one of these disciplines i Since World 
War II the Inajbr sburces bf funding foi: basic research underlying 
human factors work have ^een the National Aeronautics and Space 
Administratldn (NASA) and the military services.; Since the passage: of 
the Mansfield Amendment (Public Law 9l^AAl^ i970) tb the U.S. defense 
budget, which mandated a shift toward system development and away i^cc. 
basfc research, the real dollar volunie of research has not increabtd 
very much. What research there is has focused increasingly on 
short-term goals. As a result the basic knowledge needed to provide 
the underpinnings for human factors applications to new technology has 
not been genera^^d^. The need to reverse this trend is at least part 
of the reason that the military services and NASA have taken the 
initiative in sponsoring the work of this comlttee. This report 
reflects the committee's recommendations for needed research in terms 
bf bbth long-term and shbrt^term objectives. 

This report does hot attempt to cover the fulf scope bf human 
factors erigiheerihgi even in relation tb military and NASA heeds. As 
the committee began discussing research heeds ^ a vide range bf 
possible topics was considered. 'Two of bur meetings included tours 
arid discussions bf ongoing reiearch in military labbratbriei . 
Commtttee members were encouraged to develop brief position papers on 
highlighted topics that were germane to their interests. The human 



factors cpSonity was surveyed . through an article In the Bulletin o f 
the Human Factors Society, and 116 responses were received; th/ survey 
results confirmed the importance of many of the toplts already 
Identified by the committee. Some topics were dr<>pped. and some new 
papers were generated. Others were combined Into coherent units; 
still others were deferred for further study or Initiative. 

The material m this report 'Is the result of that process. E:ach 
chapter Is designed to be a self-contained report of an important area 
in which research Is needed. All the tOpIcs discussed here meet the 
following criteria: U) each topic Is germane tO; our military and 
NASA sponsors; (2) the topics are within the expertise of the 
committee; (3) each topic has been, in the opinion of the committee. 
Incompletely addressed by previous or current military and civilian 
research efforts; and (4) the potential results of the recOmmetided 
research will be important contributions to the scientific basis and 
practice of _human factors. And the worjc of the committee Is ongoing. 
In addition to the research areas presented in this report, work on a 
number of topics Is In various stages of development: (1) ' 
organizational context in relation to design; (2) team performance; 
(3) simulation; (A^ huian performance modeling; ^55 iulticoior 
displays; (6) human factors education, and (7) accident reporting 
systems. We expect to address many of these as well as other topics 
in sabsequent reports. S 

IS. the paragraphs that follow, the areas of research suggested by 
the committee are summarized together with some of our major 
reccsinmendatlbns. The chapters themselves provide a detailed 
elaboration of these topics. 
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• HUMAN DECiSION MAKING 

A. centr.l issue In the doderit.ndlng of haian perfor„>ance is hu^n ^ 
decision making, it has b.come even more Important With the increased 
role Of automation m complex modern systems rangix^g from military 
^ command, control, and communication systems to aircraft and process 
control systems. There has bee. n.uch support for r..e.rch on deciszo. 
making over the last 15 years, particularly by the Defense y^vanced 
- Research Projects AgeScy and the Office for Savaf Research. This • 
researth has yended to locos on formal decision theoretic 
constructions, which, while analytically powerful, have proved to-be 
insafftciently robust to. reflect the strengths and weaknesses of^human : 
, decislon-ma!;ing capabilities. The committee recommends farther 
research, with an emphasis on moving into uncharted areas. f 
- Surprisingly, despite the effort devoted to decision-making 
. research in general, there is still a need for research on how to 
structure practical decisW prbbl^s and on improving the realism 5f 
models that claim to relate to decisionmaking performance. Be do not 
know how to represent decision situations that evolve dynamically, nor 
do we have a systematic framework from which to consider decision 
aiding. • . 

Furthermore. We ire cooing to realize that many planning 
activities actually involve decision making that cannot be modeled by 
enumerating the possible states of t^ world and courses of action in 
a unitary decision matrix. They often evolve over time in bits and 
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pieces with limited central direction* ^We need a deeper understanding 
of such dfffase declislbn processes Iri orde^ to provide effective 
computer ilds for this kind of decision. 

While previous work has led to many decision-making a^s ind ' 
models, ng criteria or methbdblbgles have "BeeB suggested for 
evaluating their relative merits. Until such cpmparlsons ate made, 
practitioners will continue to advocate thel^ own products without a 

basis for choice among them. Finally, there is a persistent need for 

_- - ■ ' \ ' 

development of Innovative ways of soliciting preJerence and relative 

value dudgm^ents from people, a problem^ that leads us directly to , the 
second topic. \ 



ELICITING EXPERT ^UDGIffiNT 




The application of expert judgment covers everything from oedlcal 
evaluations to Occident investigations. Although the subject matter 
ranges wldeiy. It is our belief that thgre are generic, sobstantive 
research issues that should be addressed in a coherent program. These 
problems recur in diverse Contexts for which ellcltation methods 
either do not txist or are Iriadequately standardized across 
applications to yield consistent results. The research issues include 
I (1) creating a common frame of reference from which to assess 

jUdpiehts among a group of experts; (2) fbrnUlatlng questloSs for " 
experts in a way that is compatible with their inental structures or 
. cognitive representation, of a problem; (2^) eliciting judgments about 
the quality of Informatlohj (A) detecting and ldentl>vlng reporting 
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Was in judgments; and (5) mlhlitflzing the effects of memory loss and 

- _ ^ 

distortion on the reporting of jpast events. 



SUPERVISORY CONTROL SYSTEMS 



Supervisory control is a relatively new Conceptualization of sys^tem 
, function that is playing an increasingly important role In automated' 
systems. In such systems, operators supervise the semiautomatic 
control of a dynamic prbcess ,. su5h as a chemical plant qr railway 
- system. Typically the-, operators work in teams and Control comput^.^- 
which in turn mediate information flow^among various automatic . 
Components. Other examples of supervisory control Systems are modern 
aircraft, medical intensive care units, power plants, and distributed 
-ommand and control systems such as may be foand In military 
operations or In manufacturing by robots. Such systems deempTiaSlze ' 
the importance of human sensory and motor capablittles and emphasize 
complex perceptual and^ognitive skills'. This perspective is 
relatively new to practicing system designers; work is beginning to be 
sponsored in these areas, but much futf^her development is needed. 

Supervisory control iay be thought of as a generalization froi 
earlier work on monitoring and controlling complex systems; in that 
sense the fdundatlbris for modeling and theory are established. The 
theory ^ust be greatly elaborated and extended, however, to meet the 
analysis requirements of current and, future systems^ As the humin 
sklils of thinking, reaSbningj planning, and decision making become 



key, the models must be able to acconunpdate these hanan^^apacities and 
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limitations. This 1^ a ch61ct*opportunlty btlng together work on 
control theory .odels .tid cognltl vender,., representations. 

Cognitive psychology Is also advancing bur understanding of 'the 
vay Vn whldh resources Mre shared among varl.u. processes within the 
btMh. This woFk has unexplored Implications fdr understanding how to 
".codify system design to ch.nfee perceived, workload" particularly m the 
complex t.sks typical of Supervisory control. Each of the military 
services has research programs foc'used on humao workload analysis, m 
our opinio, many of ^em are too ■appllcatlon-o'?lented; they need a 
stronger, fbcu. on^/isearch to advince the knowledge base from whlch^ 
new application, techniques will emerge. ' ' 

Another key conc^ in supervisory control le prediction and the " 
> control of human error. Our understanding of this topic is in Its 
infancy. We have no general theory of human e^rror. although theories 
abound for hum.n response time. Human reliability analysis has been 
m vogue for several years, but. as currently practiced, it simply 
uses the numerical aggregation of historical data on recorded humL, 
failure rates.- It Is weakest In just the situations In which It Is 
most needed-when the activity involves complex diagnosis . situation. 
Assessment, and Interaction with computers. 

At the level o^f design, there are three major questions-- how to 
design supervisory Control tasks to accbmmodate human capabilities and 
limitations; how ta organic and display the Information needed to . 
carry but the'se tasks; and how much' control to delegate to the human 
versus the automatic parts of the System,. 
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^ J USER-COMPUTER INTERACTION 

Since computers a?e already playln« a major role ln)mbst new system 
developments, including lUpervliory control systems, issues of 
facilitating the learning and use by both computer professionals and 
novices has been accorded a chapter df its own. - / 
At a^March 1982 coliference on user-computer interaction, more ' 
than 100 papers addressed a variety of _ topics related to hardware and 
software design. More than half of the 1.000 participants were sysc.T. 
_ design specialists from industry and government. The committee 
believes that this.level of interest fbretiUs a heavy demand for 
scientific k>,owledge tRat has yet to be criated. Although a number df ' 
industrial laboratories are supporting proprietary work, there is only 
one raajor funded collaborative effort between computer Science and 
human factors specialists, that at Virginia .Polytechnic Institute and 
State University (fundid by the Office foJ- Naval Research). 

Most human factors research has been done in the area of computer 
hardware. Information is available on Which to base design declsiops 
concerning information display hardware and keyboards. , Mapy 
alternative input devices, such as joy iticka, brack balls, and light ' 
pens, have been studied in the Apntext of specific -pplications . 
There is a need for. further work on input devlcei that focuses on 
comparison among the full range of devices across a broad iet of uses. 
Including Instruction. tSxt processing, and graphics. ; 
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Automatic speech recognition and production have attracted Such 

interest as the technology improves. Speech as an alternative to 

manual and visual modes of input and output needs systematic 

Investigation. Fundamental work is necessary on the design of 

interactive speech dialogs tha t involve , inherently sequential 

communication and potentially heavy memory demands on the listener. 

As computer termini! are becoiicg pervasive in the white-collar 

Workplace, concern is growing about the adverse effects bri people from 

Ibng-term use of terminals with cathode ray tubes (CRTs). A recent 

study by the Rational Institute, for Occupational Safety. and Health 

found no radiation hazards from CRTs but did find a substantial 

increase in worker complaints of fatigue arid other health problems 

from sustained dally use. This study was riot able to distinguish CRT 

•design-based complaints froi those relating to the task or other 

features of the wbrkplace=-and this Is an urgent research need. In 

Europe, governments are ribw mandating standards for Wbrkplace 

designs. It will ribt be long before simllai actlbns. are taken in the 

United States arid the research must begin now to anticipate them. 

in the area bf Sbftware design, research needs are only beglririirig 

to be filled. Effective desigri of sophisticated software implies 

I — ' «t 

understanding of human knbwledge sytems and the ability to represerit 

not only what a userJkribws but also how a user makes irifererices from 

that information.^ There is i rieed for models of users' Uriderstariding 

of the systei.arf^ which they are interacting, a problem that is 

Important for eupervisbry control applications as well. 

Perhaps the most rieglected research area In computer iystem 

development Is how to produce effective materials and reference 
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irifbnnatibn. While design principles developed for printed material^ 
^re uiseful for computer system jdocumentation, there are ddcumeritatibri 
opportunities unique to interactive systems that we do not yet 
understand how to exploit^ effectively i 

Finally^ there is a need to understand in more detail the 
characteristics of the user population that make a difference in» 
computer system design. We need research that suggests, in pbrametrlc 
terms, how changes In user characteristics should be. reflected in 
system design changes. % 

The committee regards aser-computer interaction as one ^f the 
most urgent topics on which to undertake research initiatives. 



^ PGPUtATiON GROUP DIFFE^NCES 



Through public sentiment as well as government legislation^ our 
society has mandated the elimination of discrimination among 
population groups in the design of jobs and workplaces. In addition 
to racial discrimination, there is growing concern abijut 
discrimihatibh bji the basis of sex, age, and disability. We lack the 
research necessary tb describe the natur^ and extent of performance 
differences among the varibus populatibh grbups about which 
discrimination is a concern. The committee believes it is in the 
national interest to undertake the research necessary to accbmmbdate 
this reiationshp between population group differences and design. 

It is not enough to consider population group differences per 
ie. In some cases the effect of a group characteristic auch as age bh 
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performance inay depend bh the value of sdme other variables ^ such as 
Hfflount of training or level of Interpersonal skills.' It may be 
misleading to discover simply that performance deterlb'rates with age^ 
when In fact training or expedience may reverse that trend. Such 
Interactions remain largely unexplored.',^ 

/ - - ^ > 

There Is also a tiBed to understand the way in which these 

\ 

differences in performance should Influence workplace design or 
training procedures ; We know how to write equipment specl f Icat Ibhs 
designed to fit 95 percent of a particalar aser^ipopulation insofar as 
body 'dimensions are concerned, bat for most other human performance 
characteristics we lack this knowledge. 



APPLIED METHODS ' 

Much human factors work is performed under constraints 61 money-, time ^ 
arid opportunity that preclud| the use of the kind of experimental 
methods used In laboratory research. From necessity, human factors 

practi tibriers have adopted or developed a variety of applied methods 

f ' 

fbr acquiring or brganlzing information related to human 
characteristics that arise in the context of system design^ 
devfelbpinerit ^ arid evaluatibri. Examples of these methods are task 
analysis, irifbrmatibri flow arialysis^ cbllectibn and arialytis of survey 
data, evaluatibri bf physical mbck-ups^ arid the atl^uctured walk 
through. In contrast to the methods of scieritlfic research, which are 
maintained and disseniina ted in univerGlty curricula and textbooks and 
by epecialista who devote careers to improving and inventing 
experimental deelgh procedures^ applied methods In human factors work 
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a^e deserlbed only briefly ii technical project Biporti, Which .re 
difficult to access » and if forts to improve or Invent methods occar 
largely In coitneetlon with a parti^lar project. ' 

There Is a clear tieed to develop a compendium of standard 
descriptions ot the mds^ important applied methods. This compendium 
voaid be valuable for use in human factors curricula in colleges and 
universities and for continuing education tutorials for human factors 
practitioners. Currently most knowledge of applied methods is gained 
^ through on-the-job experietice. " 

^ Documenting existing applied methods, however, will hot fulfill 

------- - 

the methodological needs for all current and future system design 
purposes. Advances, in computer technology applied to automatic. „aa 
supervisory control systems anci computer systims themselves ai; have 
profound methodoiogicai implications for the analysis and description 
of the roles people play in these sytems. Existing iethods such as 
workload analysis, protocol analysis, aild function allocation require 
research to modify and extend thei? use in new applications in which 
the emphasis is on cbgnltive functions of operators r^iher than on the 
perceptual-mdtbr function* prominent in old syitems. 

Similarly, there is a need to develop new methods to provide 
information of the type and -form necessary to reiblVe such issues ai 
translating task reqUiremintsj into personnel selection criteria, 
^derivin^ training reqUiremeifti froi functional requirements * and 
describlngy^oi evaluating ;the effects of task or system functions on 
the affective responses of personnel. 

All the basic research needs addressed in this report require 
experimental investigations to provide the theory, principles, and ' 
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data to support huiin factors work In the design iHd evaiaatlon of 
systems. The application of the knowledge derived from basic 
research, however, will occur largely through the use of applied 
methods, documentation of existing methods arid research to extend arid 
Initiate methods to meet future^ needs are as essential is the 
substantive research to inJpi^S both the scleritific basis and the 

practical effectiyeriess of human factors work. 

-I ' 
\ 



CONCLUSION 



System design and the world, Of work are uhdergoing profourid changes. 
Iri a j>eriod when automation is replacing the need for finely tuned 
perceptual-motor activities by skilled Operators, human productivity • 
is no longer easily assessed in terms of Uriit output. New systems 
place iricreased: demands on the cognitive arid decision-making aspects 
of human performance. The role of people in systems is shifting 'to 
those of monitoring and directing otherwise automatic processes in 
industrial production, transporlation, iiiitary operations, and office 
work. 

These changes In humari-machlne relations both offer new 
opportunities and present nej*: ;pr6blem8 for system design. It is 
therefore tliely and appropriate that the committee's first report of 
research nee^s in human factOSs emphasizeV the importance of 
understanding fundamental cognitive processes 'and their role in 
Interactive and supervisory control systeisi 



II 

HUMAJ} DECISION- MAKING 



Work brganlEatlons, and those who staff th^, rise -and fail by their ^ 
ability to sake, decisions. These may be major strategic decisions, such 
as the deployment of forces or Inventories or local tactical decisions, 
such as how to promote, motivate,' and U:^er8tahd particular 
subordinates. To list the kinds of dec^lons, that heed tp bl Jde and 
the stakes that sometimes ride on thei would be to repeat the obvlois. ' 
Decisions are made explicitly whenever one conBClbUsly combines beliefs 
and Values in order to choose a course of action. They are made 
implicitly whenever one relies on a ritualized response (habit, 
tradition) to cope with a choice between options. Repetition of past 
decisions may result in subdptlmal choices; howeverijit may also provide 
a ready escape from the difficulties and expense of explicit decision 
making. The reasons decision inaking often seems (and is) so difficult 
are quite varied, is are the opportunities for interventions and the 
needs for human factors research to buttress those interventions. • 

One problem is information overload: More things need to be 
considered than can be held and manipulated in one's head 
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8ln.ultaneou«iy. Coping vlth sUch coipatatlon^ prbbleme is an Ideal task 
for computers, and ther* are a variety of software packages available 
that In one way aSother combine decision makers' beliefs and values in 
order to produce a recommendation. ChlSlng between and using these 
decision aids forces one to face a second Inherent aifflculty of decision 
-king: not knowing how to define (or structure) the ^eclnion prbWeui 
and to assess one's own values, that :1s. how to make t«de-offs between 
Sbipetlng objectives. Becausi analytic decision-making methods cannot 
operate without guidance on these Issues, judgment Is an inevitable part 
of the declsloh-maklng process, as is the need for. Judgmeflg eilcitsclon 
methods to complement the decision aid ^see Chapter 3). \a third 
difficulty is knowing yhen to stop analyzing and start acting. Taking 
that step requires* one to assess the quallty^f the decislon-iaking . 
process and reconcile any remaining conflicts between the re^oSendatlon 
It produces and that produced by one's own Intuitions. To help one 
through this^step. a decision aid must reveal its own limits in ways that 
are psychologically meaningful. A fourth difficulty Is that In many 
interesting decisions one knows too little to act confidently. When 
uncertainty is a fact of- life, the role of good design is to ensure that 
the best use is made of all that is known. 

The existence^ these f6a^ problems is common knowledge. Their 
resolution is cbm^icated by a fifth difficulty whose'identif ication 
requires researc^ People's commonsense Judgments are subject to robust 
and systematic biases. These biases make it dlfficu-lt to rely ok 
intuitlqit as a criterion for the adequacy of decisions and the methods ^ 
that produce them. Decision aids must accommodate these biases and may 
require supplementary training exer^^s lest their recommendations be 
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adopted only when they affirm Intuitions that are ^nown to be fauityi 

given the muitltode of decisions that are made^ any research or 
design effort that made ev^ a minute contribution to the quality of a 
minute proportion of all decisions would bear a large benefit in absolute 
terms. Proving that such a benefit had been derived would be as 
difficult as it is in most areas of human factors work. Whenever 
uncertainty is involved^ better decisions will produce outcomes only over 
the long run. That makes it difficult to establish the validity of bona 
fide improvements and easy to fall prey to highly touted methods with 
good face validity^ but little else. A sound research base is needed not 
only to develop better decisibh-^klng methods, but also to give users a 
^flghtlng fchance at being able to identify which methods are indeed better 
for their purposes. 



BACKGROUND 

Ad hoc advice to de^cision makers can be traced from antiquity to the 

Sunday supplements i Scientific study of declslbxi making probably beglhs 

with the development of statistical or Bayesian declsldxi theory by Bbrel^ 

_ _ 

Ramsey^ de Pinetti, von Neumiann, Morgenstern, Venn, Wald, and others. 

They showed how to characterize and interrelate the primitives] of a 

general model of decision-making sitoatiotts, highlighting its subjective 

elements. The development of scientific decision atds-<:ould be traced in 

, the work of Edwards^ Ralffa^ Schlalfer, and others, who showed how 

cbiDplex real-world decision situations could be int^ipreted in terms of 

the general model. Essential to this model is the notion that 

. ' w ■ ■ ■ 



ERIC 



decision-making problems can be decbmpbsed into components that can be 
pssessed iiidlvid^aiiy. theQ combined -into a general recommendatloa that 
reflects the dccisibn makers' best Interest, Those componeots are 
typically described as options^ beliefs^ and values or alternatives, 
opinions, and pref erervces, or some equivalent triplet of teris. They are 
interrelated by an integration scheme called a decisioS ^ule or problem 
structure (e.g., Flschhoff, et ax., 1981* Sagc^' 1981). 

More generally, decision-making models typically envision fou^ 
interrelated steps. 

1. Identify ill relevant courses of actiois among which the d'-cirioa 
maker may choose. This choice among options (or alternatives) 
constitutes the act of decision; the deliberations that precede it are 
considered to be part of the decision-making process. 

2. Identify the consequences (advantages) that may arise as a result 
of choosing each bptibh| assess their relative attractiveness. In this 
act the decision maker's values find their expression. Although these 
values are essentially persbhalj they >ay be clarified by techniques sUch 
as moitiattribute utility analysis and informed by economic techniques 
that attempt to establish the market value of consequences. 

3i Assess the likelihood bf these cbhsequences » being realized. 
These probabilities may be elicited by straightforward jad^entai methods 
or with the aid of more sophisticated techniques^ such as fault tree and 
event tree analysis i If the decision inaker knbws exactly what will 

happen given each course of action, it then becomes a case of decision 

_ __ • __ 

making under conditions of certainty and this stage drops out. 

4. Integrate' ail these considerations in brder tb identify what 
appears tb be the best options Making the best of what is br could be 
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libbWS it th. a«i Of the aiiiiion li the h.Ui.rk of good d.ililo. 

.eet. .1th .l.fortuBi Md iH uSSesiiea optlbh li bht.in.ai 

Tn«e steps i^i both dsBMlhg iSa v.6„e. FUlflluSg Eh„ 
co..lde«hU itlentlo. to d.t«l i„a he .eeo^pllehed In . v„i,t, o, 
.a... HO..oV.t. the, oot .ven he fo„o>^d ,.^,,UHlr. If Ihsl.ht. 
seined .t one Step l..d the deelilbh ielcef t^ revise the ^.l„u 
^ perfo?.ed .t * dlf,e«5t step. Thl. fl^iihillt, h« produSid . ,„iet, 
of „odel, .nd -ethods if d.elslon Siting Whose i„ter.elitlo;.<^if e not 
always clearly specified. 

The opportunity for roatlnlzlng and ierchandlslng t/eee 
decislon-^fclng procedures led to one of the academic L consulting 

growth industries of the 1970b * a^j^ i - \ 

19706. A wide variety of .oftwWe packages and 

flr.s can now bring the fruits of these theoretical advanc^to 
practicing dec^.lon „«.ers. Decision analysis, the „o.t coo^on na.e for 
thise procedures. 1. part of the curriculum, of .oit business schools. 
■ Although It has „et considerable initial r.^lstance fron. decision n^akers 
b.c.u.. of Its novelty a.d because of^the e.pllcltness about valu.. .nd 
beliefs that it ^requl^.s. decision analysis s.ems to begaining " 
considerable acceptance (..g.. BonczeW^t al.. 1981; Brown. « al. ^ 
1974; R.if.^ ,,,3). Thl. acceptance see„s. eV.. now. .o go beyo.d .H.t 
could b. justified on the ba*l. of any empirical evidence of it. 
efficacy. Figure 2-1 giv.. so.e examples of th. context, within which 
decision-aiding .che»e. ^elyin^ on interactive coaputir .ys,e»s have b... 
operating a.d have been reported in the prof es^.ional litirature. ' Figure 
2-2 i. .i^ilar to the .o»nary printoW'or ^one .uch .cn.s.. which offers 
physicians on-line diagnoses of th^ causes of dyspepsia. 
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Account lH8^;:hilping to assess the financtai vlUlllty of corporations. 
Clinical diagnosis-helping physicians to decide whether to perform 

diagnostic procedures and how to interpret their results. 
Counseling-helping people to choose careers or consider having chiJdren. 
Energy— choosing where to site energy-producing, facilities. 
' Meteorology— derivation of 'precipitation forecasts. 
Military— deciding whether troops are in an adequate state of. 

readiness; preplanning responses. 
Petroleum geOlbgy^^aHocation of resources for oil exploration. 
Pharmaceutics-helping in monitoring f±^d reports in order to decide 

whether drugs need to be recallbd. 
Research and development— deciding how^o allocate funds. 



FIGURE 2-1 Examples of Operating Decision-Aiffing Systems 
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ROTHERMAN AREA HEALTH AUTHORITY 

MONTAGU HOSPITAL 

SYMPTOM PROCESSING PROJECT 

i 

: ^ HISTORY SffiET 

f 



UNJT NO, 1 456/89 
SURNAME; Smith 
FIRST NAra:S: John 



CLINICIAN: Dr. Gardner 



SYMPTOMS INPUT T0» CCSlPUtER 



Male 

Age 60-69 

Site epigastric 
Radiation none 
Duration Img-lyr 
Pattern epreodlc 
Pain is moderate 
Progress worse 
i^gd by food 



Relief antacids 
Nightpain pres. 
Nausea present 

Y^?i^^°8 present 
Meals: pain Immed 
Haemateniesis abs 
No indigestion 
Bowels bk 
Micturition OK 



COMPUTER PROBABILITIES BASED DN THESE SWffTOMS 



FUNCTIONAL 
CHOLECYSTl"tiS 
DUODENAL ULCER 
GASTRIC ULCER 
CA. STOMACH 
none of these 



22 

2 
76 

e 



0 25 50 



75 
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X- 



- ■ If you judge any of the above probabiiit±»B ^n 

adjisst them accordingly. prooaDiiitles to be In error please 



PROVISIONAL DIAGNOSIS If appropriate is— 



Level of confidence In this diagnosis. 



very tentative 

1 2 



certain 
5 



The highest probability has been assigned to GASTRIC HLCFr Tf i-k. - 
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, Behavioral decision (ftigi^ «n^^^ arid Hogarth^ 1981; Slovlc, 

et al. , 1979; WaHst^n/ ^^0) ! has ;^akfc^ out of the realm 

of Mtheiatics and merchanais^pg t^i^k the^r^ of behavioral research by 
recognizing the iole of judgiaeat its st^Jpfelrig prc^lems ind in,^ 
their component^i ResearcherB- in this figid have Btudied^ in varying 
degrees of detail, the psycholdgital processes uhderiying th^se judgTC^ 
arid the ways in which they b^ improved through tirairiirig^ tasfc 
restructuring, and decision-aid design. A particular focus has b^en cn 
the ideritif icatiori arid eradication of judgmental biases. The research 
described' below is that which seems to be needed to help behayidt^ai \ 
decision research fulfill this role. i 

An important develbpmerit iri this research over the last decade has 
been its liberation from the mechaii4stic models of behavior inherited 
from economics and philosophy i The result has been more prbcess-oriinted 

theories, attempting to capture how people do make and would like to make 

_ _ _ _ . 

decisions (e.gi, Svenson, 1979). Th>8 change was prompted. iri jiart by the 

realization that mechanistic models offer little insight iritb ceritral 

questions of applications, such as how action bptibris are generated arid 

when people are satisfied with the quality of their decisibris. These 

developments are reflected in the research described below. 

There may seem to be a natural enmity between thbse purveyirig 

techriiques of dectsiori ariaiysis and those studying their behavioral 

uriderpinnirigs,^ with the latter revealing the •limit s of the prbcedutes 

that the former are trying to sell. lis general, hdwfever^ there has been 

rather good cooperation between the two camps. Basic researchers have 

of ten: dlibsi^h to study the problems that practitioners find most 

■. • - ^ -.f . . . _ ^ ' • 
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research i EBtabllshed chinnetn f» s, * 

Channels (e.g^, conferences, paper dl.trtbutlon ' 

llstsj exist for s..u,beri of this cotSffiUnlty to communicate Wth one 
.npth.r. Hany of tH. I...,n, practitioner. H.ve doctoral-lev.I training 
usually in psychoiogy. W.ge.ent science, operations research, or 
sy.teu,. engineering, and u^intain academic contacts, indeed, thi 
qo^ntity Of basic .research h.. been reduced by the diversion of pote.ti.I 
researches, to applied work. .Ith6ugh its quality .ay have benefited fro. 
being better fopused. Although problems remain, research in this area 

has a fairly g.od chance of being u^.ful .nd of being u..d. In addition. 

none of the ^.search issues discu...d. in the following Sections appear. 

to pose any serious methodological difficulties. The conventional 

experimental methods of the behavioral sciences, are iuitabl^ for * 

performing the recommended investigations. 



{ _ . . 

RESEAReH ON DECISION HAKING 



V 



ai.en tha rel«ii«lj, good" cb-.uhliiii„„ b*t»eiii dacisloi-^iiclog ■ 
r.««ehir. iSd pr«»tlcH«i. tfia pri„r, fcc».Cf the r«o».Sd«l.„. 
th.t fdlb. li the praauctl.- cf ie» -„««ch. « oppiied ^ lt» 
_^ d.s„«l«tl<,.^ It .ee.. «„<,a.ble to 6»p. th.t the i^e c.S„.i..ti<,H 
net-orl. ihit Bfioght th.« ippll^j pfitt,.. to the =tta-tl6i if 
.c.d.«.. «u «ffy ti,.i. pi'rtl.l ioiutlod. tse. t. the Held. K...irifi 
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on da^lilbh making per se asBUmei that there are general lessons to be 
learned from studying the sorts of issue* that recur In many decision 
probleiss and the response, typically made to them. In fact, the 
complexity Of real decision problems is often so great as to pteveht sbie 
lessons from being learned from direct study. ' . 

These reconunendatlons are cast in terms of research needed to 
ln.p?ove the usi of computerized aecis^on aid^. referred to generically as 
decision analysis. /These aids work in an l^ractlve fashion, askip. 
people to provide critical inpats (e.g.-. the iet^ of actions that they are 
considering, the probability of those actions achieving varioas go-lO. 
coibinlng those inputs into a recommendation of what action to t^;-, 
repeating the process until users feel^that they have exhausted Its 
possibilities. In order to be useful, an aid mast: (a) deal with those 
aspects of decision making fcr which people requl« assistance, (bj ask 
for inputs in a language compatible with how people think intuitively 
about decision making, and (c) display its recbmiendatlons in a Way that 
properly captures their implications and def initiveness . Achieving these 
goals requires understanding of (a) hOw people assess the quality of 
human performance in decision-making tasks, (b) the nature. of 
decision-making processes, and (c) how pedple assteSs tfie quality of 
decision-making processes, both those they perfoS and those performed 
for them. The research described belOw is intended to contribute to all 
three of these aspects of systems design. It is also intended to 
facilitate the development of supplementary components of 
decision-iuppSrt systems, such as exercises for Improving judgment or for 
more creative option generations 
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In this light, research that contributes to hardware's? Software 
design should alio be 'a useful adjunct to.anx formal or sen>lf orma'l " 
decisionmaking process In which Judgment plays a role. Even the devotee 
of decision analyslB often lacks the time or resources tp do anything but 
an informal analysis* 
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" Decision Structuring 

Decision making is commonly characterized as involving the four 
interrelated steps described earlier, irhe first three of these give the ^ 
^biem its structure, by specifying the Options, facts, and value issues 
to be considered a^ well .s their Interrelations. Prescriptive iodels of 
decision tnaking elaborate on the way these steps should be taken. Most 
descriptive theories hypothesize some deviation of people's practice from 
a prescriptive model (Fischhoff, Goitein, and Shapira, 1981). These 
deviations should, in principle, guide the development of the 
prescriptive model. That is, they show how the prescriptive models fail 
to consider issues that people want to incorporate in their decisions, 
in practice, however, the flow of information is typically .iyiaietrical . 
with prescriptive models disproportionately setting the tone for 
descriptive research^ 

As a result, decision structuring is probably the least developed 
aspect of' research into both prescriptive and descriptive aipecti of 
decision making (von Winterfeldt, 1980). Prescriptive models are ' 
typically developed from the pronouncements of economists and others 
regarding how people should (Want to) run their lives or from ad hoc 
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lists of relevant consideration^. Descriptive models tend more or iesi 
to assume that these prescriptilns are c6iiect. Neither Seems to have 
explored fully the range of possible problem representation* that peopie 
use when left to iheti own devices. 

Paying more attention to the diverse ways in which people 4o make 
decisions would enable decision alders to offer their clients a Sore 
diverse set of alternative ways in which they might mak. decisions, along 
•with some elaboration on the typical strengths and weaknesses of each 
oethod. Some research projects that 'might serve this end follow. 

o Studies of dynamid, structuring, allowing for iterations in the 
decision-making process, wlt^each- round responding to the insights 
gained from its predecessors (Humphreys- and McFadden. 1986). Can people 
use such opportunities, or do they tend to stick to an iniiiai 
representation? Are there initial structures that are less confining.- 
which should b(B offered by the aids? 

o Studies of goals other than narrow optimizatibii. In econoiic 
models, the goal of decision making is assumed to be maximizing the 
utility of the immediate decision, feceatly attention has turned to 
other goals, such as reducing the transaction costs from the act. of 
making a decisi5n. improving truSt between the individuals Involved in a 
decision, making do with limited decision-making expertise, imposing 
consistency over a set of decisions, or facilitating learning . from ■ 
experience. Th^retical studies are needed to clarify the conseqinces 
of adopting these goals (e.g.. how badly do they sacrifice bptisization); 
empirical studies are needed to see how often people actually want to 
accept them (partlcalarly after they have been informed of the results of 
the theoretical 8.tudies)i 
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b Option-generation •todiesi beciBion makers can bhiy chbdse 
between the optiohs thejr/can think of. Each decision heed not be a neU 
testvO^f th^ir iiagtnationB, particularly because research Indicates that 
fmaglhatldn often falls. Research can SQ^gest better fbrmuiatlon 
procedures and generic options that can be built . into decision analysis 
schemes (Gett^s and Fisher ^ 1979) * 

d ^ny decision analysis schemes are sold as stand-alone systems^ 
to be used by declsldri makers without the help of a professional decision 
analyiSJi The validity of these claims should be tested^ partlculariy 
with regard to decision structuring, the area In which the largest errors 
can occur^iPitz, iet ai., 1980). Research cbUld also show ways to Improve 
the staiid-aione capability (e.g., with better Introductory training 
packets). 



Measuring Preferences. 

Unless one is fortunate enough to find a dominating laterriatlve, one that 
Is better than all competitors in ail respects, :^maklng decisions means 
making trade*-off 6. >fhen one cannot have everything, it is necessary to 
determine the relat-lve Importance of different goals. Such balancing 
acts may be particularly difficult when the question is new and t&e goals 
that stand In conflict seem incommensurable (FjLschhbff, et ali, 1980)^ 
Dealing with hazardous technologies | f of example^ leads us dally to face 
questions such as whether the benefits of Syeing one's hair are worth a 
vague, minute increase in the changes of cancer manv years hence. 
Decision analysis schemes seem to complicate life by making these 

^ . . ' 3B ' . ■ : :■ 



Inherent conflicts apparent (McNeil- et «1.^ 1978). They actaaily 
cdnpllcate it when they pose these questions in c umbers ome, anf ami liar 
ways in order to elicit the Inforniatlbh heeded by their models— e.g., ho» 
great an increase In your p^bbablilty of being alive In five years* time 
would exactly, compensate for the .20 probability that you will not 
recover f^m the pi^posed surgery— and does this trade-off depend oh 
o.ther factors? . -. 

Such questions are difficult in part because their format Is 
dictated by a formal theory or the programmer's convenience ^ raCher than 
by the decision maker's way of thinking. They are also difficult bec-u-oe 
of the lack of research guiding their formulation. Research bh thd 
ellcltatlon'df varues has ligged behind research on the ellcltatloh of 
judgments of fact (Johrisdri and HUber^ 1977). Although there are many 
I highly sophisticated axiomatic schemes for posing value questions, few 
have been empirically validated for difficult, reai-iifa Issues i In 
practice i perhaps the most cominbn assumption Is that decision makers are 
able to articulate responses to any questldri that ^Is stated in good 
English. . 

The projects described below may help solve prbblems that currently 
are (or should be) worrying practitioners. Some similar needs have been 
Identified by the National Research Council's Panel dri Survey-Based 
Measures of Subjective Phenomena (Turner and Martin, 198X). 
^ o No bplhlbh. In moei behivorlai decision research^ as in most 
survey research^ economics, and preference theory, people are typically 
assumed to know wpat they want. ' Careful queitldnlrig Is all that is 
needed to reveal the decision malqer's Implicit trade-offs between 

- ■ _ _ - . o' 'i . 

whatever gdals^ are^ being compared. The need for some respdhse^ls bfteri 

■A ■ ^ , 

' 39 . : 
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«-sa. ...,3. . ^^^^^^^^ 

--.on ... ..e no op.,... ^^^^ ^^^^ ^^^^^^^^^^^^^ 

although they of^en show that people have a dl^^ 

- ^ ^ " disturbing ability to 

manufacture opinions on diverse fana *v *i 

^"'^ fictitious^ topics. 

o Interactive vaiii^ 

tlve value measurement, one possible response 

situations In which decision makers' vai o 

poorly articulated for 
nonexistent) 1. for the de^slon alder t. ingage In a dl r 

eHi.nl- ' engage In a dialogue with the 

^^^^^^^ 

-P"c...„. ^3..,,^ ^^^^ ^^^^^ 

«^-..«e. ^^^^ 

PeopU ^^^^ ^^^^^ ^^^ ^^^^ ^^^^^^^^^^ y 

.... ,,,, . ^^^^ ^^^^^^ . 
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^^p. . .... ..X.. .... 

lie.... ^^^^ ^^^^^ ^ 

o, ^^^^^^ ^^^^^ 

p„„... ^^^^^^^^ ^^^^ ^^^^^^^^ _ ^^^^^ 
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how people think about good and bad butcbmes that are distributed over 
time. One migh^ discover that people have difficulty conceptualizing 
distant consequences and therefore tend- to discount them unduly; such a 
tendency could be countered ty the Use of scenaribi that reify 
hypothetical future experiences. Medical counseling and the setting of 
safety standards are two other areas with specific problems that reduce 
the usefulness of decision technologies (e.g., the difficulty bf 
imagining what it would be like to be paralyzed or on dialysis, 
uriwillingriess to place a value on human life). 

o Simulating values. One obvious advantage of computerized ^v^thms 
is to work quickly through calculations using alternative valuec 
different parameters. A possible didac^c qse would be to help people 
clarify what they want, by simulating the implications of different sets 
of preferences ("If those were ybur trade-bffs^ these would be your 
choices"), both on the problem in questibn and on sample problems. Work 
along thi's line was done at one time in the context bf social judgment 
theory (Hammond, 1971). Completing it and making it accessible to the 
users of other decision aids would be useful., 

6 Framing. Recent research has demonstrated that formally 
equivalent ways of representing decision problems can elicit highly 
ihcbhsistent preferences (Kahneman and Tversky, 1979; Tversky and 
Kahneman^ 1981). Because most decision-aiding schemes have a typical 
manner bf fbrmulatihg preference questions^ they may inadvertently be 
biasing the results they produce. This work should be cbntiriued, with aii 
eye tb characterizing and studying the ways in which decision analysis 
schemes habitually frame questions. 

41 



17 



Evaluation 



The decision maker looklti^ f^r- u * 

xuoKing for help may be fiWAm*v^^ w 

--^ - . ^ swamped by offers. The range 

- ^^^^ 

^^^^^^^ ^^^^^ ^ 

.v.I„.„„„ ^^^^ ^^^^^^^^^^ 

1"8). Such . p„,„, ^ijs, ij^^j,^ ^^^^^ 

-e, ^^^^^ 

questions like: ein people seD-riF* ^ 

^ separate Judgments of fact from JudgBents of 

value? Wheh decision makers «e set to act *n 

the name of in 

en peopU .„»..p.., ..,,„„^ ^^^^ ^^^^ _ 

b. pl«M 6S th. ,u.llty.<,f the d.clslofi proi... - ,, 

F^wcess and on the qoallty of 

42 
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the butcbme that arises? What level of successful butcdmes- should be 
expected in situatibiis bf varying difficulty? This work would be 
primarily theoretical (Fischer ^ 1976). 

b Clarifying themethbd's degree bf determiiiacy. To what extent do 
arbitrary changes (i«e«^ bnes regarding which the method is silent) In 
mode bf applicatibn affect the decisioas that arise (Hogartfi and 
Makridakis, 1981)? Similarly, one would iike^'Tome^ general guidance on 



the sensitivity of the procedure to changes tn various aspects of cVc 
decision-making process, in order to concentrate ef f otts the most 
important areas (e.g., proSlem structuring or value elicitatibh) • 
Conversely, one wants to know how sensitive the method is^t^ the 
particulars of each problem and user. That Is, does it tend to render 
the same advice in ail circumstances? 1 



o 



Assessing the impact of diff erent irethods/ oq ^•"process*' var^ 



sycJi as the decision maker's alertness to new inibrmatibn that threatens 

the validity bf the decisibn analysis or the degree of acceptance that a 

_ __ 

prbcedure generates fbr the recommendation it produces (Watsbn and Brown, 
1978). Such questioning bf assumptibhs has been the gbal of mUch 
existing research, which shbuld provide a fira data base for new work . 
(although many qUestibns, such as the first two of the three raised, have 
yet to be studied). 



Improving Realism 



The simplified models of the world cthak decision analysis software 
packages use to represent decision problems are in at least one key 
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-cl.lo„ .Id. .r. ,ot the „« Of contlngeHc, pl.„oi„,J ,,.„^^ 
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such planning is anticipating future situations and prescribing the 
actions negded should they actually occur. In principle, preplanning 
responses should allow a more leisurely and thoughtful ahliysis with 
better utilization of experts and decision aids than would be possible if 
one waited until a situation demanding an immediate response developed. 
The /success of such efforts depends on the planner's ability to Imagine 
lA advance how various contln^gencies will appear should they come a^ut . 
If^tte actual contingency does not resemble its linage^ then the 
(preplanned) decisions based on that image will seem lpapprb|)rlate . In 
such cases, the decision maker must decide on short notice whether to 
adhere to the plan (and assume that his or her immediate impress:. 
faulty) or come up with a new plan on the spot (and assume that the event 

that was anticipated is not the event that occurred) i Although the 

— ___ _ _ 

stakes riding on contingency plans are often very large, we have little 
systematic knowledge about the correspbnd^ce between actual and planned 
contingencies. Research is needed on (1) when and why situatiolfe look 
(or feel) different when they occur than they did during planning and (2) 
what to do when plans made at an earlier time seem Inappropriate. 

6 Overriding recommendations. The moment of truth for the decision 

aid comes when the decision maker must decide to follow its 

_ _ 1^' _ 

recommendations or override them. Analogous moments face the users of 
y most other human-machine systems, suggesting that the study of Overriding 
would have broad Implications. The research questions are: When do 
people even think about overriding? How valid arc the cues that lead 
them to do so? How much better than the aid are thelt intuitive 
judgments? Does protracted reliance on decision aids increase or 
decrease intuitive declslbn-inaking ability? Existing research on the 
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Clearly, however, this ideaitzation dften is not realized In prabtlcei 
there may be many parties to a decision; some' decisions ^ust CYdlve over 
time (or at least are made to seem that way); btbef cleclslbhs are made by 
people who do not think themselves as decision isiakers (e.g., 
^ppervtsors monitoring and directing the behavior of subordinates or 
systems); some decisions are made by people who are not officially 
recognizable as decision makers (e.g., aides to a senior official). 
Rather different forms of re^arch are needed to Improve decision miiklng 

/ ^__;_>- ...^ , ^ y J • ' ■ . ' , 

in each setting; a nmaber of them are outlined beiot^i 

d Multlpersoa decisions. Decision theory mkthods are typic<»l^y 
designed to explore and aggregate the belle^fs and preferences 01 - 5^ -I. 
individual. One approach to dealing with multiple decision makers ie a 

computational scheme for aggregating their beliefs and preferences prior 

■ - - - - > _ 

to using them in a common decision model (RbhrbaUgh^ 1979). Theoretical 

work has suggested a variety of analytical aggregation schemes. /Although x 

_ _ _ _ _ _ _ " _ _ _ _ . _ 

this work should continue^ ,it could be usefully complemehted by empirical 

studies (using slmUlatibhs and experimehtatibh) of hbw greatly^ the 

- ___ _ _ ____ .__ ___ Q .__ 

results of these various schemes differ and hbw well they are accepted by 
users* Another approach is &d have the parties aggregate their' 
perspectives through some structured interaction (Sachman, 1975; Steiner, 
1972)^ This ap|>rpach, well worked by students of ^he risky shift and of 
the Delphi methods, might benefit from research asing computerized 
systems that allbw participants (perhaps at different sites) to go 
through many rounds of interactions with vjirying communication channels 
and protoco;i8. For example, wiiix decisions be reached more quickly and 
adopted mbre enthusiastically if the parties can observe visual images of 
brie; another^ hot Just printed summary statements? 
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o Evolving dtelslons. Insofar is dicliiSns represent cholera 
■ between alternative courses of action, any decision may be expressed as a_ . 
. statement of action ("I (or we] will lo X'J. Such twSsiatlon of a ^ 
complex decision process to its procedural Implications can have 
drawbacks. One Is that the anderiylng rationale of ad iction is lost, 
naklng It difficult to understand why things are done the way they are. 
how to respond to new contingencies, and when It Is time to rethink the 
whole decision.^ A second potential drawback Is that those decisions that 
still have to be made are not addressed directly, leaving crucial steps 
to guesswork (e.g., an operator niay be told something to the effect of/ 
"Figure odt what Is going bh and then follow steps Sj to S^"). A 
third possibility Is that procedures may have Internal Incdnslsten-les or 
be at cross-purposes, and people either do not realize It or they realize 
it bu^. do not quite know what Is wrong. ' Systeis that udd rules over time 
may be particularly prone 'tb this problem (the social security system Is 
an example). s5me combination of artificial Intelligence, decision 

- - N- ■ 

modeling, and experimental work might help people to diagnose the logic 
of the systems that they deal with and that they are called on to 
redesign (Cbrbln, 1980; Klein and Weltienfeldi 1978). 

b Unwitting decision a>akersi Just ai any decision may Se thought 
of as an actlbh^ sb Say each action be thought of as a decision. Most 
Btadents of declslbn making would probably agree wlth^the hypothesis that 
people. would be better off if they realized the declsTohs Implicit in 
their actlbnsi and Structured them bb Buch. For example^ i supervisor 
con^mpl^tlng the shutdbwh of i plant because of a malfUnctlbn wbuld Sake 
wiser choices with even m rudimentary decisior?«naiyBli (I.e., listing 
aljj possible courses of #ctlbn. Sketching out possible consequences and ■ 
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cbritlhgehcles, c^adeiy working through the expicted utility of each 
action). Such structuring has become part of the training of some ' 
qiedlcal students. The user of computerized infortnatlon retrieval systems 
Ce.g.j Prestel, Teletext) might be Usefully seen as making a Series of 
decisions (such as: These alteiriiatlves are aoblguous— whlfch gives me the 
best chance of getting the Information I ne"ed? ii it worth my time and 
money to use the system _ on this problem? Is the answer I got complete ' 
enough Or should 1 keep working?). ^Useful way to exploit existing ' 
research would be to translate It Into crude aids, adapted to the 
conditions and probi^ms of particular work settings (along with an . 
evaluation of their efficacy). : 

P Unofficial decision makers. Senior officials In many 
^organizations are too busy to make deliberative analyses of the many 
decisions they must consider. A cbnmibn (and sensible) defense Is to have 
aides conduct the analyses. For this strategem to work, the senior 
official iust communicate well enough with i'he aide to ensure that the 
appropriate problem Is ^^ressedj theWde iust communicate well ehbugh 
with the senior official to ensure that the rationale behind the 
decision-making method and the .implications of Its' conclusions are 
understood well enough to Irt'^^rly represented and afforded due \ 
consideration. Cpimunlcati^bn problems are likely to be particularly 
great when the official must present the conclusions to some larger 
public or when the training of official and aide are quite different, 
consider, for exaiple, the difficulties experienced " by public officials 
enunciating the pcllCles devised by .econbslsts or by those of junior 
executlvis trying to sell. dectsiofWlyei^^^^^^^ 

' - -'^ :_• - . _ _• - ■ t __• • • ^ _„ , ' 

execMlveBi i Batter methodc Of coSunicatlon (and for ri-allzlhg the lack ■ 
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-n.ly.is ,e.g.. draining fn„s. prac.ic. exercises) ,.,er 

compiete (Federlco. et al., 198!T5. ^ 



CONCLUSION 



.ia,„. ,„,,,,,,„^,^ ^^^^ ^^^^^^ ^ 

Of ..n.„e p„^,„ ^^^^ ^^^^^^ 

ai„e„„. ,,,,,, ^-^^^ 

P.o„i.es „. „ ,,,,, CpaMUEi... i.., .ni ie.. 

.«.„P»nlea ^^^^ 

and usable, -\ 

ihii. proj.e.. re,>.xr. p^liifii, „p.„^„,.i ^ 
^h. H.Ji.„e ^^^^^^^^^ ^^^^ 

need a context that affords teady contact ^iru a , . 

. J ^ contact with decision theorists and 

practicing decision anaivsts rv.^ 

naiysts. The fonner can solve the questions of 

-e.r, to Which th., ar. .ost .^.ed; ^the l.tt.r can provide .cc.s. to 

their machine, ^and perhaps to th.lr clients) and facilitate the 

translation frbn> researct^to practice. 
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)R«ATION FROH EXPERTS 



Many formal and Informal processes in working organizations hinge on the 
effective communication of "expert Infon&tlon." Risk analyses may 
require a metaiiurgl-st to assess the likelihood of a valve's fracturing 
Under an anticipated stress or a human factors expert to assess the 
likelihood of Its failing to open due to faulty inalnte^ance. Strategic 
analyses may require sabstantlve experts %o aiiess the growth rate of the 
Soviet economy or the proportion of its expenditures directed to arms. 
Tactical planning in iarkfetlng or the military may demand reai-tlme 
reports by field personnel "of what seems to be happening "at the front. - 
Air traffic control typlciliy Requires succinct » unambiguous status 
reports from all concerned, eomputerlred career-counseling routines or 
procedures for establishing entitlement to social benefits assuie that 
lay people can report on those aspect| of their own lives about which 
they are the ranking experts. The U.sV^Censas Bureau makes similar- 
assumptions when asking people about their employmerit status, as a step 
toward directing federal policies and Jobs programs. In produc| 



The principal author of this chapter is Baruch^ischhoff,. 
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liability trials technical experts give evidence in a highly stylized 
manner* 

Ai^^ban be seen from these examples, experts may talk to the consumers 

of their advice directly, to ellcitors who then translate what ^hey say 

^ _____ __ 

into a form usable by a computer, or to a computer; Insofar as computers 



have been designed by people, all of these communicatiqn modes assume 
some fairly high level of interpersonal understanding. The ellcitors 
must ask questions that people can sensibly answeip* The recipiehtt> ol 
those answers must interpret them with an appreciation of the errors aifd 
ambiguities they may conceal, the quality of that communication is 
likely to depend on the novelty of the problems^ the historic level of 
interaction between questioner and answerer^ and the quickness yith which 
miscbmmuhicatiohs produce diagnostic signs. Poor elicitatibn by air 
traffic controllers may become visible vety quickly; whereas employment 
surveys may (and have) elicited biased responses arid misdirected economic 
planning for years without the error's being detected; Particularly 
clumsy elicitjition may lead users to reject the eliciting system, thereby 
avoiding mistakes but also wasting the resources that have been invested ^ 
in its design; 

New research about elicjLtation and the translation of c^stlAg 
research findings into more usa^e form could benefi^ a wi^e variety of 
enterprises; -tevthis chapter discusses, eiicitation Is not a field of * 
inquiry or application in and of itself, but a function that recurs In 

many problems. This creates special difficulties for tiie accumulation 

........>..'.. . ^ ^ 

arid dissemination of knowledge about it. 
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P.rHaps because eHC.a.ion U . part o. ^ p^oBU.. ... .n of ...el 

. it has emerged neither as a dlsclpU.. nor as an area tH.t is seen tp 

require special exper.i... .,3u.p.ion is .Ha. .Uci.aU "n i« 

no. a particular problem, .s longl?>|Hin,. stay fairi, .i.,i, 

uses cbmmoh sense. The val^H^^^ 

The validity of that assumption „ay not be qliestioned 

until some egregious problem h.. clearly Arisen from a particular 
. failure. Wh.n problems arise, th. lac. of a coh.r.nt b^dy of .„,o»ledg. 
encourage ad hoc solutions. ,.ith little systematic testing or 
accumulation .f knowledge. Solutions are generated from the resources .f ^ 
those working on a particular problim and viewed fr.s thilr narrow 
perspectives 

c«pt.r U y. 

for »hlch there 1. re.p.«„Mxlj^. *,.t... ii „ 

.U*ie.t that the« enough fec.rrent tK»e. i. g.„e„t. » c.h.rtii 
.f IcSovledge. thi.ety ^ia„a„g the digree S. SSieh «ci, *y.tei 
- ...ilsner «ee. wxth eliil^ii„„ p„bli. ™.i .t.H fr» .cr.tch. ' 

«.;.»u,h w.tR i., .till focu. .petlfxe prohlS.. c™eeptu,U.l,, the. 
^ in . general i,er.iae Sith ihe p..l ,,,, 

..e B«,.th o, ^t.p.„xv. .Uh which thett «I„ti.S. .J. „t.tpt.t« iSa 
. report«,,.»eiu.e i -^p. eUsint 6f the.e prp^.et* 1. Se.lipg «th 
.uh.t..tl,e e*p*tt*, theif cmatl™ «iiet .hpuK. ,« t. giiefte . 
better nnaer^tihdlSg. Si Sh^JuagBehlil pfSceaee, Pf expert.. 

. ' §w ' ') - ■ : . 



reee&rch bases^fd^fh^ fdiiowing projects are sufficiently 



diverse thlt further detaila^jare given within each context. In some 
cases» there Is a distinct research iiteratare on which new projects can 



based. In others^ the proposed topic dpes not exlst'as a separate 
pursuit, or at* least not within the context of homan factors; the 
iiteratare cited Is suggestive of the kinds of , approaches that have 
proven useful In other fields or related problems that might be drawn ^on 

RESEARCH ON ELICITATION 



)r and 
itloir 



EKLC 



Ensuring a Cominon Frame of Reference 

Ah obvious precondition for communication is ensuring that ellcltbr at 
respondent are talking about the same thing, in ordinary conversat 
the participants have sbme^bpportunity for detecting and rectifying 
ihisuhderstahdings • If ques^bns are set €own once for ail respondents, 
then misunderfitandings must be anticipated In advance. Some implicit 
theory of potential (misjihterpretatibhs must guide the questibn 
cqmposers for mariagemerit systems ^ accident repbrt forms, or automatic 
diagnostic routines that rely on expert judgment. . - ^ 

These problems arc notj of cbursei unique tb human factbrs. They are 
probably best understood by prbf esslbnals whbse central cbncern fbr_ the 
longest periods of time h^s bc^h asking. questlbns; ' these inciude 
anthrbpolbgists (Agar^ 1980) ^ linguists^ historians (Hexter, 1971)^ 
survey researchers (Payne, 1952). phllbsbphersi and some social 

60 , 
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psychologists (Rosenthal ind Rbsnow iOfiov ^ . ' ' 

^ ^° general conclusions tha^ 

one can derive fron their work is tftt ^ ''^ . 

'^V*^ •^f'e oppojrtunltles for ' 
-»n.e.,„.„.„ ... ^^^^^^ ^^^^ ' 

Imagine intuitively. 

XiUle ,„,,-,.^^. ^^^^^^^ , 

for Uy ^^^^ .^^^^ . .. : 

.UcXt«X.„ i. ,,,,,,, ^^^^^^^^^ 

e.„sia„. . ^^^^^^^^^^ • 

■ on. X, ,000 ^^^^ 'mimi^MJs,,sim!iiji^ • 

(Bet) cod., fi,„dker.«l.f p...ief„ , co«Xd.«Mi' 

.ona Of .o«. t..t ^^^^^ _^ 

.ppa«„., .o„c.ptu.li.. ..p..«„ce.- x«...., ..e„ ..o„ .ao» 

.ucH ,„..™..,,, _ ^ _ 

on . 5.,^ .o.,..e..«, 3j„,,„ ^^^^^^^^ 

W f.c. . ...... „ cU,„, e„».U««t to sooi.1 „f . 

c...u.eH,e. ...... ausno.«, „^ ^^^^ 

. u..r.. d«crlptxon of p,e..nti„s .^Ptoi.. Th.« . 

evon -1th th. uau, «« the iost i.„„,- ' 

"th.n.h th. ..t.n. ^^^^^ ^^^^^ ^ ^ 

.I»..nt. o.n a..l,nefs to «pX.x, . J.,,., » 

res.„.h ^^^^^^^^ ^^^^ 

«.ii*i<.l. th. ..thS.. .„cH « .ntHrOpOlasy io 



uncover mlsuriderstandings. Using theie Sethbds with 8Myii5pieB of - 
upers prior to deslgSlng systeis or In the early itages of* design cbaid 
effectively suggest minor -changes or even major Issues (such as whether 
. the system could ever stand alSne, or whether It will always need an '% 
Interpreter between It and the aciaai user). ^ Such strategies are 

increasingly being used In surv^^ " design; they ^maj^>Veri leadVf o some '{ 

-—^--.--^ - - _ - - ' _ "'-t-^ ' ■ -■.,■■.„.:,'■- . 

revision in the categbrips of Justice Be^irtment -statistics so "Ss tb mi^e^^^^^^^^^ 

• thWfiofe compatible wl^h the ways ifl which, victims of crimes think about 

. ^^iryexperience (Natlbnai Research Council. 1976). 
• \ , ^ 

Another research strategy Is tb review; existing case i todies of 

mishaps (e.g., in diplomacy, survey research^ police work, or softwa're 
design) fbr'^evldence of problems due tb questioners and respondents 
unwittingly speaking different languages (Jrbbks and Bailai, 1978). Such ; 
studies would help establish the.;p re Valence of such probiewi and create a ' 
stbck of cautionary tales for educatlbrial and motivational purposes. 

A third strategy involves experimental and bbservatlbnal studies of 
grbups bf Individuals who regaiari| communicate With one thb the r. In 
order tb see how well they understand one another's perspectives. 
Software designers and less educated uiers, engineers and isachine 
operators, and market researchers and consumers are a few such dyads. 
The intuitive beliefs bf the eliaitors in_ each of these dyads regarding 
the perspectives bf their rispondents might provlde^ome prbdUctlVe . ' 

hypotheses and reveal sbme miscohdeptlons worthy, of correction. 

better ways bf eliciting Information should also suggest better ways 
of presenting It. Iiifbralhg and cbunsellni patients about medical risks 
Is one area In which these prbblems are currently under' active study (see 
apter 2). 

62 



Ha^tchlhg Questions to Mental Structurci 

^_ _ _ : _ _ . ' _ 

A presumption of many elicltatlbn efforts Is that the respondent has an 

answer to question th^t the cUcltor can raise (turner and Martin, ' 

198X)i One contributing factor to this belief; is the fadt that ellcltdrs ^. 

often fc'annot Accept "no answer" f^r an answer , needing some best guess at ' 

the answer In order tb^et on with business. A iecoBd coBtributip/ 

factor may be the tendency, long known to surveyors^ for respbiidehts to 

offer opinions on even nonexistent Issues , perhaps reflecting some 

feeling- that they can,, should, or must hav^ oplhjbhs bh ^ver^thing. A ^ 

thlrj factor Say be the eiicitors* (Intuitive or sclentlflcj mbdels bf^^ 

_ J:^ ' ^ ^ i 

>|i*g[ory that presume a coherent stbre of knowledge waiting to be tapped 

whatever questlbn prbves most useful to the elicltor (Llndley, et al*, ' i>f 
1979). 

Coping with situations in which the respondent has little or tib 
knowledge about the topic in question Is dealt j^lth In the next section, 
on how to elicit assessments of information quality. Alternatively, the ^'^j 
respdrident may have the needed information, but not In the form required *^ v 
by the questlbn. Whenever there is incompatibility between the way In 
which knowledge is brgahired and the way in which it is elicited, the 
danger _arlses that the expert may hot be used to best advantage; may 
provide aisleadlng information^ or may be se^^^ into doing a task to 
which his or her expertise does nbt extend. For example, risk assessment 
programs often require the designers bf a technical System to describe it 
in terras of the logical interrelationships between various components 



' . - 8 -■ 

■ \ 



^Including its haman operators, 'repair people, soppilers, etci) and to 
assess the probability of these cbffipOnerits ' failing at various rates, 
perhaps as a function of several variables {Jenn'ergren and Keeney, 
1981). Given these jadgmental Inputs, these programs may perform 
miraculous simoiatldns and calculations; however, the value of euch 
analyses Is Contingent on the quality of the judgments. The p^-pcesses by 
which experts are recruited may Or may not take Into consideration the 
need for these special fekllls. In sOme sitttations, no one may have ch... 

. Research designed tb improve the compatibility of questions with the 
way in ^^hlch knowledge is stored shouldTbe gUlded by substantive the: , 
about that storage as Well as practical knowledge Of the information, 
neeaed. The citations given here Represent different apprbachesjo 
conceptualizing such mismatches between precise questions and differently 

— - - .V 

organized or Unorganized knowledgi. As an example Of the kinds of 
testabie hypotheses^thal: emerge froi these literatures, consider th% 
possibility that mahy experts experience the topics of their expertise 

one by one, whereas ellcltbrs ofteh need a summary (eig., of the rate of 

--^'^--\ ------- - _ _ . - . '• ■ ■ 

target detections by sOhar operators, the conditional probability Of 

misreading an altlmfeter given a partlcuiar fiSmber of hours of flying 

experience, the distributioh of hearing deficits associated with various 

_ noise ievete)^ If .experts ar^ hot- accustomed to aggregating their ■ 

experience, then they will respond differently to procedures that request 

^'.aggregate estimates immediately and those that focus first (and perhaps 

entirely) on the Recall of individual incidents (Flschhoff and^Wiippie, 

.V 

1981). This particular research could build sOmewhat bh prbbablllty 
learning studies br atteipts to distinguish between eplsbdlc and semantic 
m<em,6ry, 

b4 ' ' ) 
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mo.. . ^^^^ ^^^^^^ 

tH. ^^^^^^^^ 

.V... ,,,.,„^ , .^^^^^^ ^^^^^ ^^^^^ 

^^^^^^^^ 

several poMlBlt peripictlvei li it i ' 
be .p,U„, ^^^^^ ^^^^^^^ ,^ 

. procedure.. ' su.K procJa.re. .,.,1^ •.i«„r.,..,. 

U,^^ ro provl^. . S..i ,,e„ f.r oj Su...e.> ior 

pro.X.« «„a, Jor .^.p,,. „^ . ^ 

.*»..,*.e eirx„.„ ^ «,or„.^,„, ^^^^^^^ 

•n* .e .elp.. ^^^.^ ^^^^ ^^^^ 

incoherent representatiyjis. 



Clarifying infSrmatibh Quality 
..f.r. J«x„. _ 

.e.. ,.e.. ... »e.. u.c.rU.S,, ,r..p. on. ro ,0 „5oo„r 

it. .ourc or tilci «t.*n.tlte 'skrie. W .ctlon i ... , . 

_^ ___ ff^ action ^e.g., hedging one»s 

be.sj. M.hough .xpllcl. asse..»...s:;of uncer.aln.y are Seco.lng^ . 
greater part of lnterprii^| i«h « rislc analysis Cr.lrley. 1977). 

weather forecasting (Hu^pfiy ind Wlniaer 1977 v . 

a wmiaer., 1977). and strategic assessment 

; S§ . 
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(Daly and Andrlole^ -1980) ^ aiuch experiences are rare for most people. As 
one yould expect in novel clicltatibn situations, th^i .respOMei tl>at 
people give are not always to be trusted. Assessments of Irifbrmatlon 
quality (or confidence or probability) have been the subject of extensive 
research over the last decade (tichtenstein, et ai., 1982).^ It has 
proTSuced a fairly robust set of methods for eliciting uncertainty and a 
moderately good uhderstahdihg of human performance in this regards The 
clearest finding is that people have a partial but not complete 
appreciation of the extent of their own knowledge. Most commonly, this 
partial knowledge expresses itself in bvercohf idehce ^ which seems qu.re 
impervious to most attempts dt debiasing, except fbr intensive trainir'^^ 

(Fischhqff, 1982). ^ : 

/if- ■ ♦ - " . 

Many practical problems; could be solved id this area with a moderate 
investment in completing the research thit has already been started^ 
This research could use the atock^.of eilcitatlon techniques already 

available tb understand better the range and potency of overconf idence 

- _ _ __ ;_ __ : « . __ ■ _ ' 

biases, tb clarify how worrisome they are, and to determihe the most 
effective training and how far it can be generalized. Of particular ^' 
interest is the extent tb which experts are prone to these problems when 
making judgtpents irNiheir areas of expertise; current evidence suggests 
that they arci but i.i is still incbhciuslve given the importance of the 
question (Spetzler 4rid Stael' vbh Hblsteih, 1975). 

The practical steps that can be taken subsequcsht to such research ,are 
developing and testing training prbceduresi idehtifyihg the least 
bias-prone elicit^tibh methbdB fbr situatibns in which training is 
impossible or .ineffective j and anticipating the extent of bias with 
different methods and situations i^brder tb apply ad hbc cbrrectiohs. 
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Choosing between these steps ahd lipliieSting theta efficiently will 
^quire a more detailed understanding of the cognitive pfocesses Involved 
In-xepresenting and Integrating probabilistic informatlOQ, Althbugh 
s^lstlng research covers n>och of the ground between bislc cognitive 
psychology and field applications. It has not quite touched bases with 
either extreme. Coping with this practical problem might provokfe some 
interesting theoretical work in the representation of Wwiedge. 



Eliciting Systems - 

In the examples used in the preceding sections, the Rhowledge that 
experts 'were asked to provide dealt with the components of some large 
system (e.g., a failure probability, a job choice, a buriiout rate). At 
.times, however, experts are required t5 describe the entire system 
(Hayes-R5th, et ai., 1981). Software packages that attempt fo elicit a 
big picture invade some'^bf . ^ei^uled strocturing, failure 

probability modeling (U.S. ifeol^^^^^^ 1981). map 

making, route pjannl^ig, and ecbndteic anal^sls.^ ^^^^ fe^^^^ have 
been programmed well enough to frork at all, one must ascertain the degree 
of fidelity between the representations they produce and the conceptual 
or physical systems they are meant to model; attempts to develop better 
elicitatlon methods or to cope with known limits or errors should follow 
(Brown and Van tihi, 1980). The research strategies outlined below, 
based la part on the initial Work already begUri and in part 6n 
discussions l»ith troubled intern elicitors, may shed soSe light on these 
problems. In" each case phe wbuld wihl to know whether a change in 



.« ■ - V 



procedure made a difference and. If so, whether one method would be 

preferred in some 6r all eitaationsi Because so little systematic 

v^' ' ._ _ _ : _ . . . : _ 

knowledge Is available oh hdw results may vary with different ellcltatlbh 

procedures^ gehera^lzihg the existing research findings should be done 
cautiously* 

6 Determlniiig whether^ formally equivalent ways of eliciting the 

same Informatlbh produce different responses* For example ^ a categbr^^^^ 

< ■ _ _ , _ ! ' ' • ■ 

events may be judged differently when considered as a whole and when 

disaggregated Into component categories. 

o Evaluating the effectiveness of methods that require more an.. 
' less "deep" (or analytical or inferential) judgments abput system 
operation. For example,^ If a prbcess produces a distribution of events 
(e.g,, failure rates)^ one could assess that distribution directly or 
judge something about /the data-generating process i 

b Varying the amount of feedback provided about how the elicited 
system operates. For example, when a simaiatipn of an industrial process 
is design^ according to an expert * s judgment . it 5ay be run a few times, 
just to see if it produces more or less sensible resuitSi The expert 
could then introduce apparently needed adjustments; Such tinkering 
should lead to successive improvements in the iodel;' however^ it can also 
prevent simulation^" from producing nonintuitive (i.e., surprising, 
informative) results. It also threatens the putative independence of the 
models created by different experts in areas such as climatbldgy and 
macrdecdnomics. The convergence of these models' predictibns (about the 
future of the economy. fSr exaiple) is used aa i sigh of their validity; 
in practice, however, econometrlcians monitor one another *8 models and 
C^djust theirs -if they produce outlyitig predictions'. • ' . . ' 
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o Asse..l.g experts* ability to judge the .ocplet^hess of a 

/representation. How well can they tell whether all i 

- . ^ wnetner fill Important consponehts 

have been Included? Avail^h^- *«4>iw ^- 

Avaxj-SDie evidence suttcestB thA9- i - 

_ »M«ge8cs cnat considerations that 

.bout . 15 . „.„ie|ii4 ..y.; tfie «.„M„ll.i, .J ■ 

a sort Of Intra-expert bratnetonnlngj. \ 



Estinating Numerlcai QaantltleS 



*lng something about a topic, bat isot 



A' common form of uncertainty Is 

v.Udy^ « .u«h e.£4.„t« dapend. Eh. .ppr<;prl.ti„ei, of the 

hms can make otherwise 
i| Subject both to external 
can systematlcaiiy 'update one's " 
out any component (Singer, 1971). 



:|ipenetrable ■ mi^^^^,f 
■^*rftfcl8m Snd;t«^.^^^ri&-Sent^:^ 
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/ ; ^^^^^^-^•r^.^^l^^ thinking and ' 

m!B^^m^:0&0^-^^ not seem to be many empirical 
W^^^%m!^m^^^^^^ .nd Makrtdakls. 1981) 
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of aigorlthni efficacy as do exist seem concentrated on the solving of* 
detennlnlstlc logical problems for which all relevant evidence is 
presented to the respondent . and a clear criterion of success exists^ 
rather than estimation tasks In which the accuracy of the es^^te will 
be unclear until some external validation is provided, tike any other 
judgmental technique^ algorithmic thinking could be more trouble than it 
is worth if it increases confidence lh';judgment more than it improves 
judgment; v > 

A primary research project here would be to compile a set of 
plausible and generally applicable algdrlthmlc str^egles. Process 
gracing of the j^^g^nentai processes of expert estimators might ^1:^ 
source'. The algorithms discovered in the study of logical problem 
solving might be another • A subsequent project cbaid attempt to teach 
people to use these algorithms, then, looking at the fidelity with which 
they can be applied^ measure the accuracy of their results and their 
influence bri cbhfldehce. The .use of multiple algorithms and people's 
ability to correct the results of Imperfect algorit^s are also worth 
study-. The best algorithms could then become part of management 
information systems, decision support systems^ and thS like. 

Two interpretive literature reviews might provide useful adjuncts to 

this research. One would look at work on mental' arithmetic of the sort 

^ ' . 

required when people must execute algorithms in their heads. Although 
computational devices shott^Ld be able to eliminate the need for such 

■_ . . _ _ _ > ^ , . _ _ _ .,,_-■_. _ K» 

exerclBes, Judges m^y^. still be ^caught without their tools or may use 

_ — ^ ^ _ _ . ' _ _ • _ 

unwritten mln±-a±^0tl|:tip^^ cbmpbrieilt estimates (once 

they've gotten thl^geriefal idea) i The second review wbuld summarize ^ in 

a form accessible to d'^igger literature on 

' 1 ;•. 
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stlSulus-preeeB-tation a.d tesponse-^tde effee'ts (Poulton. 1977). ^hat 
llt.ratur. shows the degre. of variability i. magnitude estf^tio. th.t 
ca. atise fro. "artlfactual- changes in procedure (e.g.. order of 
alternative presentation, kind of numbers used). ' • . 

^^^^ii^B Reporting Bias' 

TH. precedkS^Slp^, that .Ij|it64^.^^,;-S,^ ■ ,| 

engaged in anfe^tr unconflicted a^p£ p^od^e a be.t e.tim.t. of ^ 
Bo.e quantity^]xel^,ionshfp. :^in;^^ difflcuUii.. 
one assum^^ 5 mutpal^obd faith eff^ o^^;, pa.t'^f elicitors and 
experts to eliminate th.n>. In the real Url^, however, many *n>n8 
answers are deifberat^ their producers d^, not ^ish to have them either " ^ 
detected pr corrected; If the cit-tlons given here are at .11 
^representative, systematic misrepresentation has been of greatest 
interest to those concerned with the social and economic context within 
Which behavior takes place. Such misrepresentations may be usefully 
divided into two categories.- The first includes deliberate attempts to 
deceive in order to gain some advahtage. for ex^le, economists 
chronically mistrust verbal reports of people's p?efe?en«s (I.e.. - 
. surveys) for fear that respondents engage In strategic behavior, try^ 
to "put one over" on the questioner and distort t^e survey', results 
(Brookshire. et ,1.. 1976). Some critic, of survey ^^.se.rch are eV.n 
idvocating that respondents do so deliberately So a^^o st^p ihe survey 
Juggernaut (see Turner and Hartin. l98X). as do some people in" 
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organizations who feel threatened by computerized Inf brmatlbh systems and 
wish to see them fall. " ' 

' The second category of iisreports reflects cultural or subcultural 

■ _ _____ V 

^jrvorms. In a business or military unit, for example, optimism ^or 

_ - J\ _ - _ _ 

grduslng) may be the norm for communication between members of some ranks 

V(Tlhanskyi 1976) • Or there may be a norm of exag^erat^lng one's wealth or 
weight- Those who share the norms 4cnow how to recdde the spoken word! to 
gain a more accurate assessme^; however, mechanical system, aeslg . \ ' 
people outside the culture may take those reports at face value and 
-thereby Introduce systematic errors into tfieir workings. 

Although investigating mlsreportlng is itkeiy to be quite diiiL^.^. 
Identifying It Is part of systfems design. One way to start is to revle* 
the relevant literature In fields' that have dealt with these questions 
(e.g., sociology, economics). A secdacf Is to ^ervlew experts off th- 
record about how (and how often) they try to manipulate s^steis that pose 
questions to them. A third is" to dbSerS^e dhgbing ellcltations for which 
it is possiblle to validate responses. 

Difficulties, onceidentifiedl must still be treated. One method is 
td Institute penalties for misrepor ting . .A secdnd is to make . ct^nslsteacy^^ 
checks to detect err^l)r8. A third s to eliminate the reasons for > 
mlsreportlng (e.g., ensuring confidentiality). A fourth Is to cOrY^ct 

mlsrepbrts for knowS biases. For example^, the Central Electricity 

-_____• 3 - ' ' ' 

Gene^tlng Elpard In Great Brltatrr^discovered that it could quite 

accurately predict the time needed to return a power statidri to operation 

- - - 

by dbubling the tl-ie estiiates reported by the chief plant engineers,. 
One difficulty with such ad jQstments is that people may change their 
repprtlng practices if the^" find out about them (Kidd, 1970). 
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sufficiently to allow ree^llbratlbn of biased retrospections. Iii cases 

.. .: ' • 



0 



^ Reporting Past Events 

Many planning and design activities are heavily guided by reports , of past " 
events/ particularly accidents or other failures (Petzbldt^ 1977; 
Rasmussen, 1980), One reconstructs thi way in which a system should have v 
operated, contrasts that with the way in which it 'actually operated^ and 
aaes that coffiparisdn to improve future dSsigh (perlUps ao^signing guilt 
and enac/ing penalties along the way). ' 

tr^C^p9p*ctions are inevitably colored by the reporter's 
4cnowied8e of what has happened. As common sense suggests and the 
citations below partially document, that cdlbring can be the source of 
needed detail or of systematic distdrtidn. It has been found, for 
Example, that people seem td exaggerate in hftdsi^^ what could have been 
(and wasj known in foresight; they use explatiatdry schemes so complicated 
and so poorly specified as to defy empirical test; they remember people" }■ • 
as having been more like their present selves than was actually tKe^case- 
they fail to remember crucial acts that they n:hem8elves perfbrfefed. These 

\ \ - - - 

prbblems seem tb afflict both the garden-^iety retrbspectibns evoked in 
laboratory studies and those of professionM historians, strategic 
analysts, and eyewitnesses (Fischhof f , 1975) i » 

One needed project is to make these studies available to those 
engaged itl eliciting br using retrospective reports. Anoth^^ii to 
atte!topt to replicate them in human factors domains. Of particular 
interest are cases in which the directibh of bias has been documented 
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In which distortions are less predictable, techniques should be'develbped 

to help experts -reconstruct their view of the situation befbre^ during^ 

- -__ _ _ • « ^ 

and after the event*; For example, such research may shd© that people 

exaggerate the probability they iiiigned (or would have assigned) to. pait 

% , s - 

events before they occurred by about 20 percent, on the average. That 
knowledge tnay make It possible to adjust retro^fcctlve probability ^ 
assessments, but not to eliminate distortions In the way particular 
events and causal links are drawn. 
{ For assigning blame or understanding how an accident, situation looked 
to an operator Ju%t before things started to go wrong, strict 
"reconstruction Is essential. For understanding how the system actuaiiy 
operates, one needs to be wary of the danger that experts hSve learned 
too mu^h from a particular event, thereby ml^lhterpretihg the importance 
and genetility of the causal forces involved. Generals who prepare for 
th^list war may . fit this stereotype, as may the operators of supervisory 
control systems who respond to each Bishap by ensuring that it will not 
happen agaih^ then rest confident ttet the system as a whole is now 
fall-safe. 

Three research strategies appear to offer some promise for clarifying 
these questions. One is to review the reports^of historians. Judges, 
Journalists > and others about how they detect and avoid biases. A second 
lis to do theory-abased experiments ^ Iboking at how memory accommodates new 

nformatidn, particularly to see which proces^^sWre reversible. The 
ti^d is research on debiasin^^ '/f|o1cing' 8t the effect of directly wlrning 
people, of raising the stakes rl^dlnf^n a deci^on, or of instructing 



them to change th^ structure of th^^jtask^ to one' that uses their 

. . ^ ,_ • ; . ^ •. v.. • 

inteiiectual skills to better ad^^dhtage. 
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CONCLUSION 

f 

Eilcltlng Informatlbh from eipeiii jfdcceesf ully Is Impbrtaht to a variety 
of systems arid organizations. The care taken In ell^ltat;lori varies 
greatly,- from detailed studies of the eiicltatlon bf some specific 
recurrent Judgments, tb careful deliberations unsupported byempltlcai 
research, to casual solutions, Everi thbugh ellcltation ±b not a 
discipline per se, researcfi such as that suggested In this chapter coaid 
focus mbre attention on it and make a body" of kribwledge accessible to 
designers, in part, that knowledge wbuld be bbrrbwed from related fieidB. 
(with suitable translations); In part It Wbuld be created express^^ to 
sblve human factors problems. Soce of these projects cbiila be undertaken 
Iri their own right; others would be best dtevelbped as part bf bngolSg 
'projects, with mbre emphasis on elici^ptlon than might btherwlse be the 
case. The iriterdisclpllnary aspect of many projects may gerierate 
Interest In human factors probleis^on the part of Wbrkers Iti other fields 
(e.g., memory represeritatlon, workplace culture), and their expertise 



could cbritiflbtite tb humah ^actbrs research. 
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. SUPERVISORY CONTROL SYSTEMS 



TV 



in the past 15 years the Intrbductlon pf automation into Working 
environments h.. created more and »ore ^obs in which operator, ^re given 
vex;y high level, of responsibility and very little to do. The degree of 
respSnsibility .nd the amount of work v.ry from position to p..iti6n. but 
the defining properties of such jobs ar|: (1) The operagof ha^ overall 
responsfbiiity fot^ control of a syst^ftt, under normal oper.ti.^ 
^ conditions, requires only occasional fine tuning of System parameters in 

order to malhtain satisfactory perfonnance. (2) The major tasks are to 
^ program changes .n ih^^s or ^c^ntr^utine. ahl to serve as a backup in 



; : th. case of a failure or malfunctlcJ^' a system component. (3) 
. Imp^tant participation in system operation occurs inf^aquentiy and .t 
uni^edictabie times. (4) The time constraints associated with 
participation, when it occurs, can be very short., of the order of a few 
seconds or minutes. (5) The values and costs, associated With operator 
decisions can be very large.- (6) Good performance requires rapid • 
assimilation of large quantities of information and the exercise of 
relatively complex inference processes. 



FiLSr";' ^..thors of this chapter are 'Thomas: B. Sheridan. B^ruch 
FtBchhoff, I ,^1 posner, and Richard W. Pew." V , 
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J ; These kinds of jobs are found in the process control industries such 
as chemical plants and nuclear power plants ^ They are involved in the • 
control of aircraft^ ships^ an^ urban rapid transit systems, robotic 



remo 



te control sys|^<^ms for inspection and manipulation in the deep ocean 



and computer-aided manufacturing. They are involved in medical 
p^tient-mdnitdrihg systems and l«w enforcement information and control 



systems. As computer aids are ihtrbduced into military command and 
control systems, siich Jobs become Involved in that area. For example, 
the Army' alone currently- has 70 automated or cdmputier-aided systems ^ t 
the concept development stage (U. S. Army Research Institute^ 1>79 , iSe 

_: ... . . . . _ . ^ ^ . 

other services have similar projects undj^r develdpmeht. . , ^ 

The. human factors problems involved in supervisory cdntrbl systems 
can be ciassifed into five categories. 

1^ Disp l ay s in the pa^t these systems have used large arrays df 

^ 

meters and gauges or itf^i^.pitoation boards and control panels td display 
information, with the gi^nerai goal of displaying everything, because one 
never kn^bws exactly what will bfe needed. Little attentidn has been paid 
to the need to assimilate diverse Infdrmatida sdurces into coherent 

♦ patterns for making inferences sis^ly and directly. Today computers are 
being used mb^e and more in the control of these operations; large 
display ^nels are being collapsed into, compater-generated displays that 

- _can bail ups^t^e * i^^eded infbrmatibn bn demand • These developments in 

plQrsical ^.technbibgy ire pushing human ^factors engineers to 4^vise better 

• ways of cd^ihg.and fdrmatihg large cbllectibns bf infbrmatibn tb 

■ --V^ — - ^ - - -^i y_ _ '-_ 

^ICtirlEate inftetpretatioti and reliable decisidns by operators. Also 

H needed are better means of accessing ixifdrmatti^n, d^ahs that are^hbt 



>K .... ■ . ... -; A' 



sitaationi. * 7 

j^' . . ■ • • 

operator. «„ e.«v.,, l„.ir„,,,„ . 
^ ,mn, „..pi„ « erlS.H,.or pr.f.ra„c... ...^^ 

people and iingulstlfc ityiep. ' - 

. iv^SSak-If:^. »e .1.0 l,c. «n-4.„i=p.<, S.thodol.si„ 5or 

»P*r...r a^e.p„ i TOi. fia. .1.0 ...» called .fie . 

op«.»rti.3^e.li^ piewf,j,.orpr<,blei,.p.ce.) iworfect- 
opera..,., ioael. „« l«d ai.a.tro., r..„I.. 

oi«=„.l, s Satiar of u.^l. i.pori.See for op.»«ora of ^ 
.»U.r, cosaoa and eo».r.. .,.,e.. .0 .c,„lr. pjppar .«„c.pt„.I iode„ 

«d .ceap tfi., „^.,., s...„i.s^^„^„ ,aai. u... 

Ai_Womjjd. «e h'„a „o go«d SriSapt.'.. j^.tgn 'for . ' 

opar.»o„ IS •..pervl.„r, .o„tr.r-.V.ieSi^'l„ par.Veea^. i,« p^Wd ' ' 
e«ra„.l, difficult « iaa.ore or e«i.t. JS. „S„l iol^f . ^ ^ 
involved. H,a, .^d 6. a,«, .^,,,,„,. ' _ ' ^ , 

Wrxn. ..a „ ..traSal,^*.a„df., ..a. .c£fo„ A ■ , 
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'"^^ - consensus oE Vh.. '..p.al workioid^ .r 



• ^ficlency a nd^^., i3#ues ol traipfi^^proficienty 
.-in.ena,ce ar. Ci.ical in thl. .l^^^t^S^i ^^ve..*., 
la. «o.e unl,ue and 1, d..«. ftb^ a^^r^^y .arge .at .of : 

possibilities, Bbst of which will /• 
^ ' \ ^" "'^^"'^ ^'^'^.^"8 operating life ■ 



Of tte^ysteft,^ It^ not easy to .Anticipate what ty pis of errors wlii 
occa^ ^or how to trJ||||^to prevent them. ^ 



SUPERVISORY C^tmh \n DiFFEite APPLICATIONS 



^ - This section, adapted frp^^heridifi (198i^Hrovldes brl^f comparisons 
and contrasts a^ng different application of supervisory control 

rol, and Sanipuiators . • 



systems\^ process fc^trol, , vehicle 




3 



I 



I 



rbc&is Gfintrbl 




The; t»rm process usually ref efs ^o a - dynamic syst^^ sucV as ^^^fdf«Ti 
^ "^-^"^"^^^^^^^^ g^eraHnjg plaat br ^iiemical or' oil pjodul^n 



Jacillgfy th^ is fixed ^ sp^ta, a 



and^ope^teW^ore less cbij[|in^pusly 
. ^ yiS'picai^ time consta^a^r^ tfiS^-many mit ^ ' 

< elapse after a cbh^ol altion iS taken before most^^^he^ syste^^ 

le struct4ire8 with^li 



is c 

- > 



^ \ Chi ^ 

Most siich processes Invb 




one 




ete • 

ij|«d6 flftwing^ 

ace-rb another and ^^lxej$^|^ _ai»j^f heat ^ergy to affect thg 
:d Of vlce^ versa. Typltail^ sMph sjrstemsf ^n^oive fflultiple^^bnni 



4L 

and mulj^iple mac 
location of tfife , 
rbbm vhex^e many 
^ ariJ^ other devices 




and a t lea^ some of people move 'f rbm^SH^ 




5^ 



tq^^nbther_* Usually* there is m cen t*.i*conrfdl _ 

: ■ ^ ■ r- 0% . ^ -"^ 

Blgnais are dlsplayefd and where vMlves.fcpumps, 
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fi«8'beeii emerging eiementl^^r^cess 
JiSrl^l for clecades. starting ^ith:|^roiecha„i.sl controller, 

or cohtrpl^st^tlorfsjhat could be ju«t|g Wl^4er«pr t. tnalntain 
J certain va^^.blel wltl,i^ii^f,3V(a home thermostat Js an^^amplej, , ^ 
,,8pecial.^lectronlc circUlta gradually rej,licei the'^iectrpmechanlcal 

f unction. ^iri su^ ^ 5^,^, ^ .pntrol " 

. loop by swrtihin'g to m^nuai cont^S. U8a«liy>^,ch cbdtrol statlan 

^.plMy. bbth the^iabie b.i«g'con^^^^^ r^ tempi rat drefo. , 

^ the thermost^) anj the c^htrc^ ii^ if.^.^^^^^^^ 
^urrlice). ^ cbntVol devic^ „a/|^i„ed Up in i^^l^ 

«ntrol r6bm.Vogpther'with manual switches an/^^s. |tatUB ^ 
- d^is and recording displays. 4nd a^ taany^ as l^SO^alaris or 



annunciator^^I^do^ tfia* Ught Up ind^^te what^^a.t" Vari^le kas^'| 
JU8^ gon. above or. l,elow ll.its- From Sfte^^tt,^i,-bf their aiaSs^l^. 



5* in the ilrst minute of a^^ss^^^lW icH^nt ind in the . 
second minute, by recent ^t. in a large new nuclea^'pjant) the 
.operator is supposed to divine what ±fcMpp 
The large, penet^l^purpbse.v^onipute^^^ 
control, instead of^ultlpleAikde 



3uh(f its way into prbcpss 




iM^dent. :^hy^^i6iial ' 



proportionai-integral-derlvative co^rolle^or each ^aft.blejth^ 
computer^ can'^rea^^h^ .et of variables as ^tor and ti:^ 
c.r>trol trajectory^t would be optimal Cin/tl^nse ^^st, mo.t 



f^^lls*^ or whatever criterion is iiporiEant). 



inore 'interactions tha 




•ilKals' andst^ number bi 
Ope ra t(^(_w^ ibpu t^'o 



■ r of variables ,^£h/^i 



if^use there: are many 



sslbli a^lust-metits 
comfeter-cont roller 

t 




iety of dlsplal 

s th€*liamab / 1 



thi eventa^ MUe Island, to' dev*aop diapUys that integrate 

cpmplex:_^tt0^ i,f.,rmation^ t^^ operator to-tssue ..nu^.ds ' 

; In.a^^taJ^ efficlSht. ahd reliable manner. ' Thfe t^rm' syst^ ^ . .. " }' 
J-r^,^£pis a fi^l^nable. way to desctibi th^ display of j^n lmal^ ^ pf ^ ' 
^.Ar^fM^^or, (using G* A. Miller^-s ^«U-kn^. t^r^^^^^ 

^ meaning about the current state ^^ectbr of yarfebles, '^m^^^kr^^0f ' 

the past, and,whete^±t is\ikkly to go in the, near fi^M^ W^^ Vfflff ' 




Vehicle Cdiig^rdl 



5 ' 



4'" 



Unlike the processes de^criberf a^v^i vehltoes ipve through s^at^ ^rt^i 
CJ^rj&their' oper^prs with them oftare controlled remgteiy^ V^^-- ' 
types Of vehiclls have come under f significant Hegreg^ tupe||lfey ^ ' 
-control. In tftPlast 30 yiaroi ' * 



We -might ^a 



Wth spacecraft -beca-u^^nl f sense , tieir f nftjtion-i^v 
.the^sim^est. Tfey are laun^ed to perform weil^lned ^ikioSs. and 
^their interaction With th^^^iro^ent, (othe^'^Ssigr^ity r M Stl. in^^ 



s« oifSt^l 




other wo?d^-, there are* oi oBStlfeles and no unpredictable tr»i flc to jro^ry 
about! It, was in "spScecraf t, especiaiiy«^^poll<>. ^t^iit hiinan operfe^ who 
were highly iskilled SohtiSupofi ^ntal cogtrtri (test pilots . or "Joy - 
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i^y^") '^^^^Pt. to a c|ppieteiy fiew w^of getting ■< ~ 

nformatpn from^he vehicle Aid , giving c^^ds^his'niw wa^ W^s 
the cq^piter.^ The astr^uts had . to l^rn '^'use a -g^^ . 
key^rd with prdgrams (different functions appropriaC* t^-dif ?&retit 
mission ph£'8e>5/n^un9 (operands, ot di$a«:b fc^W^'sse.d^ proceed) 

rmix^^ri^^e i^nst^ 

/ ■ ^ ... 

<: f r 'M. — ^ 




Of ^rsci the ^istrtfnauts sttit perfortaed ijprtaln number of 
contlBuous contrft function^ Tl^y controlled jthe brlg^tatlbn of the' • ^ 
vehlcil and^ inana^ivered It to accoiplish star ilghtlsg, thrust j 
?endezypa8f and lunar landing. But, as is not generally ^r^^tfted by 
j^. fcjie public, cferitroJ. in each of these^mdd^s was fieavliy a|ded. Not; only 

y- were tSS ,^rinai contror loops themselvei stablllri^ 6y electronics," But , . " 
_ "also -nSnmanaal, Sutotnatic contfbl func^ w*re fclng, s|4DUltarii-bi^ .' : ^ ^ 

Ixe^uted and •coordinated with' what the astr5^^^ " ' 

In commercial and military al?craf t*J}Jere has been jnjjre and more ' 
*' ■■ , ' ■ .• ■ . "-^s/i •. 

-supervisory control in the last decade^ oft twd. - Goiaiercial piio^s are 
galled flight ' manaaer& .^i^et^tive of t"he fact that 4ey must aiipc^^f*. 
- A ^^"^ attehtibn among a la?ge number of separafre, but complex ' 
c^puter-to^ systems. ^liUry aittraf t are ballad flying compUt«rs . 
arHfei^eed tlie cost of t^^^P^j^ lo; t^ now fit exceeds th^^st A ^ ■' ' 

- ... - - - ■ ■ •#;^^5^'.<^*_ . ' , '>i^>«y"- ■' ^\ • 

. ^rof thfe baSic^rframe*. By fiefaw^r inertia 1 oeasarement, •a,featfUte,.pf- • - 
^' - the new jumb^ jets 'as well; as of- i^itary .»ircraf t , Uhe computer can ^a^-' • 

a vsnicl^^td ahy.latit^dei Ibttgitude, aSd iltltude within a/ftaatirbn^ a ' " ^ 
klj*meter. ^^In addition tlfere . arg pany other supervisbry cbapiajid modea' , 
_ intermed|itl between such high-l^?l co^m^s' ai'd the ^^Igj^s^level tf 

pure conOnuous„ cbntrbl/bf ^ ailerons , elevators ,*^4/^hi-U8t . A^lbtican 
- set .the autbpilbt to. prbvlde ^^diaSie^ of a^sinooth conmand cofrse at 

f ixe^ tprn br. climbjrat^s to fo'ilow ii^Sn^or M^^^ ^ 
sieved to thfs <MSurs^ Thek^autopilot t^be aet to |fH^^ a ni§/^ - ' 
.^tl^ode OQ a newli^dini^ pilot can ibck onto ^d^beamySr ra^^^^^^^ 
• • Blgnal^^^foi-auto^iJ^^ the Lockfieed L-loil, fbr M^ple.^^^ f 

g^^&i^e. ttfrrat M i^ver*^if.-Gb\trjl. li la ^ / 

.*ijMwlifit*-K^-th^^^ot i^hav rfeiiable means bs/p^akfn^ ^otft ofc^th 
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>>uftdmat^^^ control modes and reverting to manual cbhtrdl or some 



Intermediate^ mode. Tor eMmplei^ when In an automatic landing mode the • 

' pilot can either push a yellow bottbn on the cbhtrdl ^yoke of jefJ\the 

_ - ^ ' __ ^ ^' - - 

yoke back to manually get the aircjrvjf? ^ack lirider direct cohtrol. 

Air traffic control poses iifter^sting supervisory ^ot^bl prpblems. 



for the headways .(spacing) betwe^u aircraft in the vicinity of major 
cbmmetcl,al airports are getti^ng tifehter and tighter, and efforts bc^b r 

iV - ^ ■ ^ f ' ■ - 

savi fuel\and tiff^vo^d noise ov^r densely, populated utbaii areas re&uirs 

■•^^ , . ^ . ^ . 

more radical f'^etBbf^hd landing tra^(r|^gr£es .WNew compateg-basegf 



commanlcatioq aids^wlll gupplement p^lv'^efbal cbmmuiiijbation l^* ^ 
pilots^ and ground cbntrbllers^^ iid'nai- display technology will h^ij 
already ^verloadbd grbum-<fbhtr^y^^ mbhitbr what is happeqiljig 

_ _ j • . ^ ■ - 1 ■■ "^V^ -~ - ' - r 

three-dimensiorial Bpaee'^ oVer largkr area%^^^rbvldlhg-pred of 




of 



f 



^iiision and related >it>il: Information., The CDTI jfcockidt' disDiav 

affic inforMtiqn) is a ^liew computer-b^sedy^icture weathejr, t&TBtxsS" ^ 
^razayj^l^h as mcJuntainS'^ and ta^ll structulrfeSi cburteJ^ff rmitibh^^cH^^ 



j ...... ^ ^ ..... . _ . 

way^poiitts, radio beacons aiT,d 'inf^kers ^ aiyl runways and cbmm 

^ P^^^^S® «s weilj[fi^he po.^i-^li,,^ 

A position) e-t^her airc^-aJ^i ^ItflfciM^es the.p*' 




riaiiy wh^h. ^Juf-the^fndow visibilfT^ 
tly^iiips ^nd submarines have beiet^oconve 




oi^trbl • 



:t manual control 
♦ 



.-.^dieted 
Indent biar ground 



sufficed for m4hy\:yea'w h^s (^een^-repiated^bath W!^ttki in 
j[iAV±gs£ioi,J&^ call* jfoT '^dmptit^r control aSd p 

^iPble^ im ^^^"^ tbwajqd high 



capaji^ lit y^ne ve r l^^^^ 



re 



ld9g t>ime lags ^prbduR^ ^ larger 4^^^^. ^^h^ 
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New autopilot and ^omputY-base^ display aids, slmllar^tb those Ife 
aircraft, are how being used In ahlps, 




. Hah Ipu^a tors and Discrete parts HaniliSg 



.V 



^. ^fiicle corit^l. Thar Waxilpulator. base may^e carried oh a s^^^fif a ;^ 

. gr^^nd'vehlcierd^ i'subniarlnt^or i^base may beflxe^ TF^ |^d » _ 
-^4Sri£^er. end effector) Is. ni|ed relativ^tp^ the base InW^ three 

oegrees of ^ransf nf -ffsti ntsA t-K^:^^ ^ - i^:^ : . = ' " - X ^ 



tftgr^ee of jfransiatllsn fnd 

-_ ' ' \ - « I 

degree of freedom f or^rlpp 



V special cuttlpg. 
,^^#^*i6ing^ or bther functions 

- Manipulators -are being uied In inar 

- j - - . ; ^ . ^ 

■'Ai^ ^ ^ lunar moving vehicles ^ undersea opera 




'^^^^ff :t^atl^n. It Say have oiie 



^ome hands hai^e' 



rehtiaj^iy 

f ''U"^ - 

ees of freedom to ,pe0fprm 

» ^iij^ng , "^liiiit: ^^ylrig^^ 



4/ 

ff erent ^i^^ii^t ioM ^^&kJ^iT\s 
and hazardous operations*' in 



InduBtry. The type of supervisory tcntr^l affd Its juMificTtlon' pfe^ 
accor^llng to the apRllcitidn. . ^ N t i 




ting from ^und^trip radio trii6inl66|bn f roi ea^tt^e^k; to 
la^a^iltlj^ unless an operator ^ts^ three' seconds after e^^h^ef -a 
/ seriei of 'iS^remehtal njovemehiB.^^^s a^es direct manual cont 
fme-con^jnlrt^nd isjjrad** cai 1 She^MT^d^^reflTc^ 



cdntror ^bbp r 



having a cots|>uter oh^Tfe tc^ ^onplete 

. .movement tas^^^^^^ usl^ local sen^"^ 
control. They propo^r^ c 



afid Ibca? 'compute 
tg this pbde supervisory' cbhtrbl 
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f^**"^ would be urilmpdrtaht ^ 



^long.^|^^^ocal-coHtrc,a.Vbuia; i^^^^^^ to deal with 

1 oJ^Bt^^sai^ other s^lf-protectlbh rapidly ^ 



£l.e 

supervisory control to He ander'sea vehicle 
manlpulator-^lso compel Itig., THere are things the operator cannot 
sense or can s^nse only with great difficulty «nd time delsy (*.g., the 
jnud may easily be stirred up. producing turbid opaque water that prevents 
the video cameri from seeing), so that local sensing and quick respor^se 



may be more reliable. For mono 



str^ictures, or ship huiil o^ suipfeying the ocean bottom to'f^a c 
object )^he operator cannot remain alert for long; if adequate .rtlfici. 
sensoxscbiild be provided for the key variables, supervisory control 
should be much more reliable. The human operator jiay jiave^her things 
to ddi so' that 8ppervis5ry^ control Would facilitate pe^^^ififi^^ks to 




tasks (e.g., Inspectingf^sl 



ilpt.-.Ti 



update^ tfhe cbmpUc«r psrograia or h^ip the remote device get 
^ i^oobie. ^ final reason for supervisory control, an^'-bftin the'most /- : 
acceptable, is that, if communications^ plwer, M other systems fallf^ 
- there are, fail-safe control Sodes Intq which the remot^ iy^teS reverts to 

get the vehicle back to the surface or-othewlie render it Vecoverable. 

_ _ _' _ • ' ' ' '^ 

Many of these same reasons for supervisory control apply' to other 

_' __ • ^ 

^^uses of manlpqjators. Probably the greatest current interest liT^ |^ 

f manipulators Is for manufacturing (so-called Industrial robots), =*' . 

in(:^udlng iachlnlng, -welding, paint .spraying ^^^fef treatnent^surface ' 
cleaning, bin ^cking, parts feeding punch^esses, ^andllng be tw^^^ 
transfer, lines, isseibly, inspection, lading and ifiloadinB finiih»d 

units, and warehousing. Today ^epetlil^e ^skn such as welding 'aftd toalnf' 

A ^ ^ ?S. ^ ■' ^ ^ 

spraying can be programmed 
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gupe^K^sor^ then Impjpmented With the 




ii 



control loops that repott positioa and velDclty. if the parti conveyor 
ii sufficiently reliable, welding or painting nonexistent objects seidpm 
• occars, bo that more sophisticated feedback. Involving touch or v^ioS; 
^ is usuaiiy not required. Manufacturing assembly, however, has proven £5^ 
be a far more difficult task. " ^ • 

in contrast to. assembly^lne operations. In which, even if there is a 
mix of products, every task is prespeclf led , in many new applications Vf 
manipulators with supervisory control, each new task is Unpredictable to 
. ^oiisldeMbie extent. Some examines are ilning:, earth moving; building 
■construction, buliding and street cleahfng and maintenance, trash 
collection, logging, and crop harvesting, in which large forces and' power 
must be .applied to external ;^bject8^. ' The human, operator is necessary tb 
program or otherwise guide tile nanipi^iator in some degrees of f%Mm. to 



accomodati? ekh n^w '^lt^a*loft,• in*^ther respfects c^rtiln'^haracteristic 

' °°**°°S^* ^^^^'-^""'^ Initia^^at the».cbrrett - 

tlAfe. ,_llh f^^'f,^/i^l^^^^oi8,, such' as im:rollrge^»j the goalJls to 

^^^i^'^^^'^'^^^^jf^P^^^ and. forces, |to extend the su^ec^''^ 
fiand'tboi t>i^pu|p^;^ tJ obtain tissue s^mpiis, 

to femove unheilthy HIsL^ or to stitch; .Again., the surgeon ^bntrol^ ' 
siie de^ees of frefedbm^fe^., o^ an op^icat^oWfe jor * cluterlzing 
■.#nare), 
pressure) i 



i 




/ 



avjtomatldn t^rols'othef varlables|Pe.g.,-alr^r water 



Theye^^^ a number of limited theories and methods In the human factors 
•ilt Mj^ H^^that should be bjrou^'K^to bear on ^he use of supervisor 
' cbrtil|^^^pstem8. ^ A great deaj^emalns \o he done, hdireveVs to a^ply them 
in this context. The disttusslon that follows deal$ with live aspects cf 

■ . \ ; - . ' ' * /. / . _ _ ^ * ■ .' , ■ ■ 

^ the problem. Tfre first conBders^ur^ent fomtal models of supervisory 

. ____ ^- . --- - ' 

control* The: second discusses display an3 c^mmind problems. 1}ie thlr^l 



taftes up computer knowledge-based ^system^^nd^-thefr relation to the 

'.; ■ . N _ /__ _ __ : _ 

Intern^ cognitive model of tjtie operator for on-line decision making In 



y 

^ _ _P^vi_^_ 

supervisory control. The fourth deals with mental workload » stress^ ftnd 



research aft«>ntiah and resbuic^ allocation as they relate to 
Supervisory cbntrbl. The "fifth is cbncerned .With Is^ti^s bf hui 



^ system irclltbljLlty, 



J 



i trusjty^nd ^ifltimate authbj|'ity . 




MbjglSling Supervisory Cbntrbl ' 



In the area of real-time monltbrihg and contrb4.^^1ttfCbntinuous' d^2^^^^ 

" - - - - - -£ ■ ■ 

, proce*e8, the jo^itlinal cotitr&l models (Barbtt aiid Kl^lnfflan, 1969) .-de<ScM.be8 

' _ ' . _ : . ; , _ f . : \ _ _ _ _ . ; 

the- perceptual ffi6to^r. behavidr' of cibm^d-loop systiims having relatively 
shSfT^lme c<^stani^u Experlmefrtatio^ oq. this topic ^as oeen ll^tej 
. yf^ggesting that th^S^ais of model may^e brol^dened^ ^^repres'ent 




- -\- 



mbiritJC^rirnip<^^^ dynamic sys'teflBlnj which 

ax 




itroX ia ^axrequent (Leviso'n'^^tsi4^ Tanner, 1971).' .There/ar^ 




^tt..pt,_to exten. t.^.^.. .o-explor. Us Applicability ^8, ior. cc^pl,, 

An lncreailng nombet of iup^^^pttdi^ystems^^ ^ '.: ' 

r.pr.^intea by a hle^^y of three ^n^ .f Interaction (Sherld.xi, \ 
19825: (1) a human ^tor Interacting vlfh a high-level computer, 



low-level computer. interacHng with physical entitles in the 
.nvii:on.ent. and C^) the resulting multilevel a.d Multiloop interaction.- 

• .having interesting .y^etrical p^ertl.. (Figure 4-1). slnc^ there" are 
three levels of int.lli^^nce (one hum.n, tWo artificial), the .llication". 

cognitive i.d co.^tional tasks^pong the :thre^ beeo.ei c.ntralf 
« ^ing RHsn=ussen^.;;ca979^^^at^oriaatibn of behavior into kno..l.dge-based . 

•^based^ and skill>based behavior, ihe .p.rator.pay a«ign rule-based ^ 

• ta^ (e.g.. pattern recognition, ron^lgg pls^nlng ^ predictive ^.odels^ 
org^izing) to. high-level computer (Figure 4.2).^ ^ixnilarly, j" 



or on-line proSr^l^jg, of the cdmpiat ' 
^eha^llor of t^Bys^^m "for 
sary^^) tnai^^v^d j 

f «n»<^^^rnfeg'g'on> e^^te^qg 



i 
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1. titk is obiervtd dirtctl^ by htirnih bpiritbr. 

2. Sj*J t_ob_»#r¥td_iod ir«ct ly th rough ^hib fi, 
^?P_meotfrs^ind_dtfpl«yt: TWi SASJiidtMck 
intwcti with th«t froin within HIS, 

3. task Ift controlftd within SM iutbmitic modi 

4. tptk ii af ftctvd toy tb« proc^^l ^ihg / ' 
•enMd. 

6. ta»k afftcts ac^uatoa and turt: »f*s<^r*d, 

6. Human optrator dirsctty sff«c6 lai^-. 

7. HujTVLn oparaior affacti ssk indfrictly 
^ntfoU^_H_lS compatau, snd ictuitbri, Th^* 
control inttracts iwith thst from within SAi, 

8. Human op*rator gtti fMdbSck f^--a^ 
HIS. ; ^ 

9. Human oparator idjitftt^^r:;-^ z^':.-:- -^'--.r:. 
10. Human optrator adlc^ ailpli / pariT >ir 



■Mteractibti in a Supervisory Control System 




FIGUte 4^2^ Multilevel 
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Display and Commaud 

Design oi integrated computer-generated displays is not a new problem, 
and the military services and space agencies have pioneered development 
in this area for aircraft and various command and control systems. B^t 
the technology continues to create more possibilities. Operate- 
supervisory control systems need to have fewer displays, not mor^=, 
telling them what they want or need to know when they want or 
know it. An additional design ^ro'-lem is that what operator^ 
need and what they really need may differ* 

As computer collaborators become more and more sophisticarod - usi 
type of display would tell the operator what the computer knows an^ 
assumes, both about the system and about the operator, anc wr.at ic 
intends to do. 

An important source of guidance regarding the design of displays has 
been and will continue to be the intuitive beliefs of experienced 
operators. The designer needs to know how much credence to give to these 
intuitions* Too little attention may mean forfeiting a valuable source 
of information: too much may result in inappropriate designs that fit 
untested folk wisdom (a pilot's belief in the value of verisiir^il itude in 
displays is an example of the latter problem). Ericsson and Simon's 
taxonomy (1S80) of situations in which introspection Is more and less 
valid is one poiat of departure for research. Studies of metaccgnition , 
people's understanding of their own cognitive processes (as contrasted 
with current psychological understanding), are a second (Cavanaugh and 

97 



1950s and - (Co ^c'^prQ; ^ 1968) are p thj^o. These GtudieR foun^:! thr:t 

n the c o u r d r .' > ^ i r o f a f; rt o n ^ c ? ' ■': .--i n ? i ni ,^ ,p, :' n e t ^. V ' ^ y r .'O y 

A.;. L^hour^'i croor -.:^ . . ■:c>i^r' j r- - e ne ".h l ed speec-. Is bolh avcfilv^b'.e o.nd 
-^^r^:-, /^ni^ n^'.thour:'^ ; ^ c r: ^,:'v^ "-.^erators v;c;rn:rv?s enc: ether in f orr:a i 
;j ■ * h u: ^''-v^'^ -r.'. ?T ' ' "l: . * . ^. ; c to :. t. 1^ :';:•? :. " .v: i v-r 

...... .-r : ■ ^ : '-..v- '-.^/^r : yrt. :' "i ~ o r^i ^ o ry cc-;:to1 . 

"ccc'iirr or -:or''^^. :'-^r , '^votcr"^. A tig re -pTi:?*!'.!' ::'>^"t it 
.ts found f.n the nev auuCp^A oc "7;nLT.and rystG^rs rlri aircraft. Giving 
coTTir^-ands tc 2 control Ky^*:f»'^. by* ?, cf *.\r:;.-,5 of 3;,^.bo'. s 1*: ryntia.v i 

« new «::^^?n:e for r.icst opc^^otore. Prcc;rc r:-': .''.r tpiyc: ^-n^ depends on c-^^^eful 
technology tr-2n'^fer from cat? prcciessinp cA'iat is self -paced tc dyaamic 
control In wn.lch the pace is detertnined by many factors. Naturaliess Ir 
use of such language is also sn iuiportant goal. 

ComiDand J in tnany circumstances, is not a solitary task. The operator 
Tiiu^^t interact vit^ ~r;i:y ~ :* :' vJ d i in cc-r to get a Job cone. Tnin nay 

n:i^ t "• "ularl".- t'"^e Cr^Ji^e vbcn th-; '^';'::j':'o the e:jer,.;ency rcecn- th.^t the 
t^rhr^cal svsteu cannot b^: trt:f-ted to re~o^^ and resnonc rel in ^1 y ^ t "^^ c 
is. aTi inters. c ting human systeuj insy assuiDe (and perhaps interface vith) 
some o.f the func.tionG o:^ the inter-^ctin/ technica'. sypte-^ Th-^ k-^.r'cs of 
h'.;~-n int?raction r!OSs:;blr :incl"j^*!c r?que-t""-n . ^or^^r^^ io". . uioni ^lori:;'; the 
'".'^ ^■'^r --e of the 5VF;te^, no tif yln.v: o-jt>;. :^r:^ ''e^p-^ for <^ vf;cn-= ^ t on , ':;o 
nrovxde sp^ci«l skills), pnc term:^ na " iny: \,n- eco : rv ccrnrrunAC^i^.ionp - 
^Tv.,_, -70 the~o interact in-rr i^:t. ':int^''? ^':;V vrl:. d ere caop? 

^8 
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.oa = .re. of ,,,ual .y.te.s „aL. s„ch Int.: vent i„.. „„r. , , „ 

/--b;., „„„ , „„, ,^ ^^^^^^ ^^^^^^ op„«.„.,, 

-tio„, ..,.,„ ^^^^ 

InCvldual (or sroup) .,o„i<, b... „o.Uor for an. cope vUh crises. ,r 
--ici„ It is .i„.,s asso.e, 33.0 i„.ivi.„al .as exper.is. 

1« boch diagnosis acd tre...an.. Perhaps i„ su,,,rvisory concrol 
.he .,uivalcr.c f„„«i„„s should he s.p.rarcd, and difiereht .ralr... 
..»pera.ar.£ called for i„ „onitorir.g a„d 1„ i„ te , vent lo- . 



Cocputer Xnov, ;:dse-Based Systeos 
ind the Operator's Internal Cognitive Mode 



- r.ot a new idea that, ir. performing a tasit, peopi. ...enow r.tresent 
tKe tasK in their heads and calcol.te vherhar. given certain constraints, 
cloing ^ „in „s„lt in j^. i„ch ideas derive fro„ anti.uit,. 



Huna.-i-Machine Control 

in Che 195QS the development of the "o.se.ver" In control systems .heo.y 
for.ail.ec s Idea. That is. a differential equation .odel of the 
external concrolled process Is included in the automatic controller and 
is driven by the s..e Input that -drl.es the actual process. Any 
discrepancy between the output of this co.p.terl.ed model of the 
e.vlron.iental process and ac.ual process is fed back as a correction 
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to the Intern..! n^cdel to force its variables te be continuously the same 
a. the actual process. Then any and nil state variables as represented 
(observed) In the Internal model ir.-,y be .-sed directly control the 
rrocess, if direct n,ea surec^c-n t of those same variables in the actual 
environment r.ay be costly, difficult, or impossible. This physical 
-ealization of the traditional idea of the Internal ^odel probably 
provoked much of the current research in cognitive ecience. 

Running in faRt-tlne, updating initial conditions at each of s 
succession of such calculations, the -^odel becomes a "predictor display" 
that provides the ooerator vith a projection of what will happen under 
given assu-.Dtlons of Input (Kelly, 1968). Furthez ccaparlsonR can be 
mnde between outputs of .uch real-ti:.e Models tun in the computer and 
those of the operator's own internal :.od,.l , net only for control but also 
^or failure defection and Isolation (Sheridan, 1981). Tsach has 
developed a realization of this as an operator aid for application to 
process control (Tsach et al . , 1982). 

Ideally the computer should keep the operator Inforired of vh;, t i t is 
as-a:.ing and computing, and the operator should keep the cor^puter 
infonacd of what he or she is thinking. 
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cognitive Science 



In the last several years cognitive PoychcKpgy has contributed soi.e 
theories about human inference that make the application of 
knowledge-based systems particularly relevant to .vn^rvlsory control. 
The idea is that reasoning and decision making consist of the developing 
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ana s..arcniny of complex problem space. (Newell ,nd Sl-.n. - -';>, 
-p;-- lying or.e or n:or. inf.r.nce procedures about lnfcn.a:lor, 
i--vledge base that represents the a^clsion ^ak.r'a un.:er- ..n.:..:., ... 
-::uation (Collins and Loftus. 1975). This i. similar to but .n-vr. 
inclusive and less well developed than the internal process c^.d.! 
control theorists. Rasmussen's qualitative -.odd oi hunun 

decision making about pr.c.s. control is e.tirel ■ ...patlble w, 
view. And, the contribution of speci.-sLs . r. artificial Intel, 
concerning knowledge-based systems provides one way to .^ple^o- 
co.-.puter portion of such hunaa-cocputer interaction. 

A number of human factors pr ..ems relate to people's ubi.. 
in nlnd the basic workings of a ..c-..plex systea. and to update tr..r . l -.. 
•^--'P-nd.^ng on the current State of the system. Recent studies of 
cognitive processes in skilled operators such as taxi driver. ' : . 

= , 1980) or chess players (Chas^ and Sinon, 1973) begin to .provide t::c 
kind of lnfor=.ation that will be neodt-c by human factors cosigner, 
evaluating these Issues. For exan.ple, how can people best be trained to 
^^^velop effective problem spacesV Wh.t is the optical mix ot analog and 
digital representation? How can the .onputer's data base system be used 
to aid the individual in developing and updating of such an internal 
njodel^ What iseans can be used to ensure that the current state of tr,e 
model fits with the current state of the system.? With what fre,ueucy 
should a person be interrogated about his or her current view of the 
r^odel to make sure tb^t he or she is still "with if in control of the 
systsc-.' For hua..:: supervision 'to be really effective, a detailed 
understanding of how the h.^an controller grasps a complex system ot any 
moment in time and updates it over time is necessary. 

101 
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How can we ctenvAv.e a given operator's Interna cognitive model of a 
Riven task at a rjven z •? One method is to 3.sk the operator to express 
it in natural language, but the obv>:o,is difficulty Is that each 
-■oera tor's expression Ib unique, nking it very difficult to cQeasure 
e-'. ler discrerancy from reality or to compare across operstors. Verbal 
vrotcccl techniques ( La 1 nbr i dge . 197^) make use of key words and 
r-l^t.lons. More forxal psychometric techniques (inultiattribLrte utility 
.nc rpe.r^er.t , con joint or multidimensional scaling^ interpretive structural 
r^odeling, rolicy capturing, and fuzzy set theory) offer some promising 
vays of telling a computer one's knowledge and values in structural torn:. 

A likely (and perhaps comnion) source of difficulty Is a TDisTT^atch in 
thf- rnental ir.odels of a system of those who design it and those who 
operate it* Operators who fail t recognize this disparity are subject 
tc unpleasant surprises when the systetn behaves in unexpected ways. 
Operators who do recognize it may fail to exploit the full potential of 
th- system for fear of surprises if they push it into Uiifamiliar 
territory (Young, 1981). On a descriptive level, it would be useful to 
un: ^rstand the correspondence between the roental models of designers and 
CDcrators as well as to know which experiences signal operators that 
^rhrre is a mlsruatch and how they cope with that information* On a 
practical level, it would be useful to know more about the possibility of 
iTr.proving the match of th^ce two sodels by steps such as involving 
operators more in the design process or showing them how the design 
evo.,ved (rather than giving them a reconstruction of its final state). 
The magnitude of these problems is likely to grow to the extent that 
designers and operatore have different training, exnerience, and 
intensity of involveaent vith systems. 

■I 02 
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ihe concept of njenlal workload as discussed in this section is not urviq-.'o 
to su^jervisory control, but it is sufficiently important in this contv^xc 
to be included here as a special consi aeration. 



Hurr.an-Machine Control (This section is adapted i rom Sheridan £:r.c 
1962) . 

During L..: "^HSt decade "mental workload" has become a concept of . - l 

controversy, not because of disagreement over whether it is i - : r l^. r. w . 
but because of disagreement over how to define and measure i c . Hili^srv 
sp' . i f i Cci L i ons for mental workload are ne^ Cirtheless being prepared by the 
A r Force, based on the assumption that mental workload raeasures will 
preJ ic L--ei the r at the design stage or during a flight or other 
c pe t ion--wl-ie ^her an operation can succeed • In other words, it is 
believed thac measurements of mental workload are more sensitive in 
anticipating when pilot or operator performance will break down t:ian are 
conventional performance measures of the human-^jia chine system. 

At the present time "mental workload" is a construct* It must be 
inferred; it cannot be observed directly like human control response or 
system per forma nc.-- , although it might be defined operationally in terms 
of one or several or a battery of tests. There is a clear distinction 
between mental and physic.:! workload: The latter is the rate of doing 




trechanical work "nd expending calories. There is consensus on 
measureraen*: s bnsed on r^jspiratory gases and other techniques for 
uieasurin physical workload • 

Cf par;, icular concern are si tuitions having sustained nental 
workload'^: of I^ng duration, M/nny aircraft irj c^i^lonb- conri^jue to require 
Fuch effort by the crew. But the introduction cf computers and 

u t on.! on in irsny systeirs haF^ come to ncan that for long periods of time 
opo^at: ors have nothing to do — the workload me.y be so low as to result In 
bor'^dom and S'^rious decrenent in alertness. The operator may then 
^v.-'-'cenly be ex]^ect?d to observe events on a display and rnake critical 
J,..,' .T'nen 1 8 — indeed. e%'en to detect an abnormal Ity. diagnose what failed^ 
't-^c take over control fro~ the -^-utonjatic TyStcni. One concern is thot the 
operator, not being "in the loop,"" will not have kept up with what is 
voing on. and vill reed tinjo tc reacquire that knowledge and orientation 
tr make the proper diagnoses or take over control. Also of concern is 
t-^at at the beginning of the transient the computer-based information 
vill '^e onaque to the operator^ and it will take CG\nc tiae even to figure 
ou . hov to access and retrieve from the systerr. the needed inf orrria tion . 

There have been three approaches to measuring mental workload* One 
approach, used by the aircraft nianuf a cturer s , avoids coping directly with 
TP^easu regents of the operator per and bares workload on a tas!-: 
tln^e-line analysis: the more tasks the operator has to do per unit of 
times the greater the workload. This provides a relative index of 
workload that characterizes task deraand, other factors being equal* It 
says nothing about the irental workload cf any actual person and indeed 
could apply to a task performed by a robot* 
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Scale, there bein£ qnmo 

, see consensus tnat "fraction of tot.l tl.e bus.- " 
■•co.nU.ve co.,.e...,.- ^^^^^^ ^^^^^J' 

c.a..c.e..s.cs o. .en.. ..^..oa. an. .n.. ... ...^e . 

xarge .nen ..e o..e.(s) are s.aU. rhese sc.... .ave .een . ^- ^ 

.i...ar, services as well as aire... .anuf ac c.rers . ^ crl.l-- . 

tha, people are not always go.,-, ^.d.es of their own ablll., • 
m Che future. Sone pilots .ay ..,,e the.selve. to .e ,ult. ^ 
sustained effort at a Mgher level when in fact th., 
Th. third approach is .he so-called secondary task or r...r.. 

capacity technique. In it a n^^•.^ 

... a pilot or ODerator is asked to alio, 

whatever attention is left over from the nri.a^v r..>. - 
task. 3, generating rando. numbers, tracking a 

«c.3en With . «.all ^tick, etc. Theoretically, the .^..er tne 

performance on the secondary task, the .esa the tl.e ..cuire^ 
-erefo.e the 1.3s the .entel workload .f ..e prl^.y .ask. . .r.tl... 
ox tnl. .ecnnl.ue is tha. l. i. i,,,.3.ve. it ... itself reduc^ t. 
at.en..on allocated to the prl^.ry task and therefore be a 

fielf-contaminatlng measure. And in r^al f^^.u. „ 

-viu, real flight operations the crew ma v 

not be so cooperative In performing secondary tasks. 

The fourth an. flna. technique, is re.l., a whole cate^orv of 
partially explored P...l M 11 ties-the .se of physiological .easur... 
Many such measures have been proposed. Including changes in the 
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electroencephalogram (ongoing or steady-state) , evoked response 
potentials (the best candidate is the aLtenuation and latency of the 
so-called P^p^^^. . occurring 300 Tni llisecondc after the onset of a 
challenging srimulus), heart rate variability, galvanic skin response, 
p . ' -f "I ry c5ar:f^ter, and frequency spectruiD of the voice. All of those 
'^'-^-ve provec to be noisy ~.nd unreliable. 

Both the Air Force and the Federal Aviation Administration currently 
h.^-.o irajor programs to develop vorkload measurement techniques for 
^:'rcraft: r^i 1 o t ^. n.;-^ and traffic control. 

If PiVi operator's mental vorklcad appears to be excessive, there are 
several avenues for reducing it or cccpen&a ting for it= First, one 
should examine the situation for causal factors that could be redesigned 
to be quicker, easier, or less anxiety=producing . Or perhaps parts of 
the tssk could be reaFsi,p;ned to others v.-ho ar ^ less loaded, oi the 
procedure could be altered so as to stretch out in tiise the succes-.^ion of 
events loading the particular oper.?tor. Finally. It may be possible to 
rive ^_71 or part of the task to a coraputer or auromatic system. 



Costnitive Science 

It ir i r-^rlp-nt, for purposes of evaluating both mental workload and 
cognitive .models as discussed :.ri rhe previous section, to note that there 
has been an enorisous change in model.^ of rnental processing in both 
psychology and computer scr'e^-^. In their recent paper, Feldman and 
Ballard (in press) argue that 

7 nc: 



Cor.teiaporary couiputer science has sharpened our noCionB o:* v^na^ 
is "computable" to Include bounds on time, storage and o-ner 



computational models ana cognitive science be at least 
plausible In their postulated resource requirement. 

The critical resource that ±h most obvious is tiue. NeL.iv 
whose basic conpu; a _ ^...via^ speed is a few si i 1 iseconds mu8: he- 
made to account for complex behavxcrj wnich are carriec 
few hundred nilliseconas . . . (Posner, 1978). Tnls me.inr:, ;■, . . 
higher complex behaviors ar-L^ carried out in less than 
time steps. It may appe.. . :hat the problem posed he.-c 
inherently unsoivable anc ..\at we have made an error i". 0'__- 
formulation, but recent results in computational comclejnry 
theory suggest that networks of active computlnp. ele.ii.'- r > v 
carry out at least simple computations in the rG:iUireu time 
range — these solutions involve using massive numters of ^^nits 
and connections and we also address the question limitations 
on these resources* 



There is also evidence from experimental psychology (Posner, 197c) 
that the hL.rr.an mind is, at least in part, a parallel system* From 
neuropsychological considerations there is reason to suppose tnat a 
parallelism is represented in regional areas of the brain responsible fcr 

different sorts of cognitive functions , For example, we know that 

different vis^^jl maps (Cowey, 1979) underlie object recognition and that 
"o pa rate portions of the cortex are Involved in the comprehension and 

production of language* We also know more about the role of subcor::ical 

and cortical structures in motcr control. 



resources . It does not seem unreasonable to require tnar 
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The itudy cf mental workload has simply Dot ke? 
advances r. the conceptualisation of the human mind 
Subsysterr. The iDajority of res enrch-ers of huraan 
t'he Interferen ,^ of one complex task with another, 
i^vid>-^r.ce in the I. ^ t ra t ur e that such Interference d 
th p c>eneral i nt er f erence nay account for only a ste 
v/-!rl a nee in totpl worklof^o. More linportant may be 
F c 1 f 1 c cognl t i ve sys terns sh-^.i ^-d by tvo tasks * In 
''-'I cks (197 8 have recently f orir \^la ted a theory of a 
-■egree of facilitation or Interference between task 
d "r-.tance between thr^ir cortical representation- Th 
-ay be m a r e 1 y m e t a p h o r i c a 1 , since we do not know w h 
the actual physical distance on the cortex or wher'^^ 
relative inter connec t ivi ty of cortical ar#^a; the la 
reac.onrr'" ^ 

Viewing h nr^i a n s in terms of cognitive s u r: s s t e m s 
ner^pective on mental workload (see Ksvon cind Gopher 
unu.s j3 1 for any hurran task to involve only a si ngle 
to occur at any fixed locat ion in the brain . Hos t t 
sensory TEodality^ in central analysis sys terns , and i 
F-y stems ^ There Is need for basic research to under £ 
neparability and coordination of such cognitive syst 
t^^sk analysis that takes advantage of the nrw cognit 
to ask how tasks distribute themselves among differ€ 
^and when ner f ormance of different tasks rn.^y draw on 
system. There is also an obvious connection between 
approach and analysis r-f irJivldual differences base 
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lnfor:.atlon processing concepts, and .uch needs to be done to lln. 
analysis of Individual abilities to the ability to .-e-.ha.e act.vlcy 
within the sa.e cognitive syste. or across different systems (LanJ^.n^n: 
Hunt, 1982), 

An emphasis on separable cognitive systems does not necessarily .ean 
that a more unified central cc....lling syste. Is unnecessary. i.deed. 
widespread Interference between tasks of very different types (P 
1980) suggests that such a central controller Is a necessary aspccv , 
,.,,3n performance. There are a number of theoretical views adc..,. 
the problem of self-regulation of havlor. particularly m si.. 
situations. Two principles have . . applied by factors 
engineers: The first is that attention narrows under stress. Thus, .ore 
attention Is allocated to central aspects of the task while Ics., 
attention is allocated m more peripheral or secondary a^^^.i^. 
Son^etl.es this principle has been applied to positions in visual space, 
arguing that peripheral vision is sacrified u^ore than central vision 
under stress. The degree to which the general principle applies 
automatically to positions in visual space or to allocation of function 
within tasks 18 sln:ply not very well understood-but it should be, A 
second principle of the relationship between stress and attention 
suggests that under stress habitual behaviors take precedence over new or 
novel behaviors. The idea Is that behaviors originally learned under - 
stressful conditions tend to return when conditions are again stressful. 
This view 18 particularly issportant with respect to the process of 
changing people from one task layout to another. If the original 
learning takes place under high stress conditions while transition or urs 
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under relatively low stress conditions, a stressful situation may tend to 
reinstate the responses learned in the original configuration. 

Recently cognitive psychologists have begun to take into account 
emotional responses produced under conditions of stress (Bower, 1981). 
One development eisphasizes links between individual differences in 
er^otional responding and attention (see Posner and Rothbart, 1980, for a 
review), AJLthough it Is a highly speculative hypothesis at this time, 
this work suggests that attention may be viewed as a method for 
controlling the degree of emotional responding that occurs during 
stressful conditions. In particular, differences in personality and 
ter^reratnent may affect the degree to which attention and other mechanisms 
art? successful in managing stress. These new models relate emotional 
responding to more cognitive processes. They have the potential of 
helping ue understand more about the effects of emotion and how it may- 
guide cognition and behavior under stressful conditions- Since this work 
has just begun, there are few general principles to link the emotional 
respons^<i to cognition as yet* Developments along this line could be 
useful for hunan factors engineers, particularly those involved in 
training and retraining and those Involved in mangement of stress under 
battlefield conditions. 

For the most part, this discussion has been from the viewpoint of the 
overloaded operator. For much of the time, however ^ the operator may be 
underloaded- In the field of vigilance research- which is concerned with 
human behavior in systems in which signal detection is required but the 
signals are infrequent and difficult to detect, a great deal Is knowxi 
about exactly what parameters of signal presentation affect performance - 
The signal detection model (Green and Swets» 1966) has been shown 
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liseful in analyzing guch behavior. Again, its applicability ha.- r.c. ^: 

evaluated in iDore coi^plex tasks in which signals are represented bv mo.e 

complex patterns of activity as would be the case in supervisory conrro-:. 
systems of the types described above. 



Huu.au Fioficiency and Err-^r: Culpability, True^:, c;nd Ultimate At..ho:-:r 

Designers of the large, complex, capital-intensive, high-^r i sk-o f-^ -• i . ^ • 
systems we have been discussing would like to automate huEnan or ' . 
out of their systems. But they kr • :hey must depend on them tc .1 
prograni, monitor, step in when failures occur with the automation, a-a 
ger:eralize on system experience. They are also terrified of hunar j: r . 

Soth the commercial aviati on End the nuclear pcver industrlc:j .nrd 
actively collecting data on human error and trying to use it analytically 
in conjunction with data on failures in physical components and 
subsystems to predict the reliability of overall systems. The public and 
the Congress, in a sense, are demanding it, on the assumption that it is 
clear what human error is, how to measure it, and even how to stop it. 

Human error is commonly thought of as a mistake of action or judgment 
that could have been avoided had the individual been more alert, 
attentive, or conscl en tious = That is, the source of error is considered 
to be Internal and therefore within the control of the individual and not 
induced by external factors such as the design of the equlpTnent. the task 
requirements, or lii-k of adaquate training. 

Some behavioral scientists Esay claim that people err because they are 
operatlr.g "open loop'-^withuut adequate feedback to tell them when they 
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are in error. They would have supervisory control systems designers 
provide feedback at every potential misstep* Product liability litigants 
sometiEses take a more extreme stance — that equipment should be designed 
so that it is error proof, without the opportunity for people to (begin 
to) err, get feedback, then correct themselves. 

The concept of human error needs to be examined. The assertion that 
an error has been connnitted implies a sharp and agreed-upon dividing line 
between right and wrong, a simple binary classification that is obviously 
an oversimplification* Hunian decision and action involve a 
multidimensional continuuE of perceiving, remembering, planning, even 
socially interacting. Clearly the fraction of errors In any set of human 
response data is a function of where the boundry is drawn. How does one 
decide where to drau the line dividing right from wrong across the many 
dimensions of behavior? In addition, is an error of comnission ^ (e=g., 
actuating a switch when it is not expected), equivalent to an error of 
omission, (e.g*, falling to actuate a switch when it is expected)? Is it 
useful to say, in both these instances, an error has been committed ? 
What then exactly do we mean by human error? 

People tend to differ from machines in that people are more inclined 
to make "common-mode errors," in which one failure leads to another, 
presumably because of concurrency of stimuli or responses in space or 
time. Furthermore J as suggested earlier, if a person is well practiced 
in a procedure ABC, and must occassionally do DBE, he or she is quite 
likely in the latter case to find himself or herself doing DEC. This 
type of error is well documented in process control, in which many and 
varied procedures are followed. In addition, when people are under 
stress of emergency they tend more often to err (sometimes, howevei , 
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^-llK^ly6ts may assume that operators are aware of an emergency when rAie^ 
are not). People are also able to discover and correct their own errors, 
which tn^vv surely do in many large-scale systems to avert costly 
accidents . 

Presumably the rationale for defining human error is to develop nit^ans 
for predicting when they are likely to occur and for reducing their 
frequency (Swain and Gutman. 1980). Various taxononiies of human error 
have been devised. There are errors of omission and errors of 
comission. Errors may be associated with sensing, memory, declsi 
making, or motor skill* Norman (1981) distinguishes mistakes (wci 
Intention) from slips (correct int-^r-^ion but wrong action). But 
present there is no accepted taxonomy on which to base the definition ci 
hiiir.an error, nor is there agreement on the dimensions of behavior r.hsz 
should be Invoked in su-^h a taxonomy* 

There is usefulness in both a case study approach to huuian error and 
iii the accumulation of statistics on errors that lead to accidents* Both 
these approaches, however, require that the investigator have a theory or 
Ljodel of human error or accident causation and the framework from which 
to approach the analysis. In addition tn*pre is a need to understand the 
causal chain between human error and accident. 

One has only to examine a sampling of currently used accident 
reporting forms to realize the importance of the need for a framework for 
analyzing human error. They range from medical history forms to 
equipment failure reports* None that we have examined deals 
satisfactorily with the role of human behavior in contributing to the 
accident circums taoces • 
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Furthermore, for accident reports to be useful, their aim needs to be 
specified. There is an inherent conflict between the goals of 
understanding what happened and attempting to fix blame for It. The 
former requires candor, whereas the latter discourages it- Other 
potential biases In these reports Include: (a) exaggerating In hindsight 
what could have been anticipated in foresight; (b) being unable to 
reconstruct cr retrieve hypotheses about what VaF happening that no 
longer niakes sense in retrospect: (c) telescoping the sequence of events 
(making their temporal course seem shorter and ncre direct); (d) 
exaggerating one's ou-n role in events: (e) failing to see the internal 
logic of others' actions (from their own perspective). Variants of these 
reporting biases have been observe^ elsewhere (Nisbect and Ross, 1980). 
Their presence and virulence In accident reports on supervisory control 
systeins merits attention. 

In addition to these fundamental research needs, there is a variety 
of related Issues particularly relevant to supervisory control systems 
that should be addressed. 

In supervisory control systems it is becoming mere and more difficult 
to establish blame, for the Information exchange between operators and 
computers Is complex, and the "error," if there ever was any. could be in 
hardware or software design, maintenance, or management. 

Most of us think we observe that people are better at some kinds of 
tasks than computers, snd computers are better at sonse others. 
Therefore, it seems that it would be quite clear how roles should be 
allocated between people and computers. But the interactions are often 
so subtle as to elude understanding. It is also c-"ventlonal wisdom to 
say that people should have the ultimate authority over machines, au^ 
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again, ir, actual operating systems we ust^ally find ourselves :.ll prepar. 
to assert which should have author, ty .r-r what circumstances and for 

how lon^. 

Operators in such systems usually receive fairly elaborate training 
in both theory and operating skills. The latter is or should be done on 
sl:.ulators. since In actual systems .he most important (critical) events 
for which the operator needs cralnin^ seldom occur. Unfortunately there 
has been a tendency to standardize the emergencies Ccla.-:sic stal. 
engine fire In aircraft, large-break loss-of-coollng accident in nucl...- 
plants) and repeat them on the simulator until they become fixt-. p. . 
of response. There seldom is emphasis on responding to new. u.. 
emergencies, failures In combination, etc.. vhlch the rule book never 
anticipated. Simulators would be especially good for such training. 

A frustrating, and perhana paradoxical, feature of "emergency- 
intervention Is that supervisors must still rely on and work wlch systems 
that they do not entirely trust. The nature and success of their 
Intervention is likely to depend on their appraisal of which aspects of 
the system are still reliable. Research might help predict what doubts 
about related malfunctions are and are not aroused by a particular 
malfunction = Does the spread of suspicion follow the operator's mental 
model (e.g., lead to other mechanically connected subsystems) or along a 
more associative line (e.g., mistrust all dials)? A related problem is 
how experience with one malfunction of a complex system cues the 
Inte. pretatlon of subsequent malfunctions. Is the threshold of mistrust 
lowered? is there an unjustlfl-d assumption that the same problem is 
repeating Itself, or that the same Information-searching procedures are 
needed? How is the expectation of successful coping affected? Do 
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operators assume that they will have the same amount of time to diagnose 
and act? Finally * how does that exi^erience generalize to other technical 
systems? Do bad experiences lead to a general resistance to innovation? 

A key to answering these questions is understanding the operators' 
o'v^n attribution processes. Do they subscribe to the same definition of 
hiiTT^an error as do those who evaluate their performance? ^t gives them 
a feclin^ of control? Kow do they assign responsibility for successful 
and unsuccessful experiences? Although their mental models should 
provide some answers to these questions, others may be sought in general 
principles of causal attribution and misattribution (Harvey, et al.. 



Supervisory control of large, complex, capital-intensive, high-risk 
systems is a general trend, driven both by new technology and by the 
belief that this tscde of control will provide greater efficiency and 
reliability* The human factors aspects of supervisory control have been 
neglected. Without further research they may well become the bottleneck 
and most vulnerable or most sensitive aspect of these systems* Reseach 
is needed on: 

(1) How to display integrated dynamic system relationships in a way 
that is understandable and accessible. This includes how best 
to allow the computer to tell the operator what it knows, 
assumes, and intends* 
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(2; How best to allow the operarcr to tell the computer vb'f: ar 
she wants and why, in a flexible and natural way. 

(3) How to discover the internal cognitive model of the 

environmental process that the operator is controlling and 
Improve that cogni'.ive representation if it Is Inappropriate* 

(A) How to aid the cognitiv*:- prccess by computer-based knowledge 
structures and planning models, 

(5) Why people make errors in Rystem operation, how to minimize 
these errors, and how to factor human errors into analy.- ' 
system reliability. 

(6) How mental workload affc . , . hiiman error making in systeni^ 
operation and refinement and standardization of definitions and 
measures of mental workload. 

(7) Whether human operator or computer should have authority under 
what circumstances . 

(8) How to coordinate the efforts of the different humans Involved 
in supervisory control of the same system. 

(9) How best to learn from experience with such large, complex. 
Interactive systems * 

(10) How to improve communication between the designers and operators 

of technical systems* 
Research is needed to Improve our understanding of human^compater 
collaboration in such systems and on how to characterize it in models. 
The validation of such models is also a key problem, not unlike the 
problem of validat^n^ socioeconomic or other large-scale system models. 

In view of the scale of supervisory control systems, closer 
collaboration between researchers and systems designers in the 
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developn^ent of such systems may be the best way for such research, 
lucdellng, and va 1 i da ti on to occur. And perhaps data collection should 
built in to the nonnal--and abnorinal — operation of such systems. 
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USER- COMPUTER INTERACTION 



INTRODUCTION 



Electronic computers have probably had a more profound effect on our 
society, on our ways of living, and on our ways of doing business than 
any other technological creation of this century. Computers help manage 
our finances, checking accounts, and charge accounts. They help schedule 
rail and air travel, book theatre tickets, check out groceries, diagnose 
Illnesses, teach our children, and amuse us with sophisticated games. 
Computers make it possible to erase time and distance through 
telecotinnunications, thereby giving us the freedor: to choose the times and 
places at which we work. They help guide planes, direct missiles, guard 
our shores, ar.d plan battle strategies. Computers have created new 
industries and have spawned new forms of crime. In reality, computers 
h.ave heco^e so intricately woven into the fabric of daily life thivt 
without them our civilization could not function as it does today. Small 
wonder that all these effects have been described as the results of a 

ccEiputGr revolution • 

GantE and Peacock (1981) estimate that the total conipiiter power 
available to U.S. businesses increased tenfold in the last decade, and 
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thai it expecten to do;jble every two to four years. According to 
iLusr recently available estimates (U*S* Bureau of the Census, 19?^;. 
there are currently about 15 inillion computers, terminals, and elecrronic 
of fi.-e mdchiiies in the United States- That number is expected to grow to 
about 30-35 million by 1985, at which time mere will be roughly one 
compute r-based machine for every three persons employed in the 
whi te-collar work force • Spectaci*"! ar advances in computer technology 
fiave made this growth possible, decreasing the cost of computer hard-^jr^ 
at the rate of about 30 percent a year during the past few decades 
(Dertouzos and Moses, 1980). 

Computers are still not as widely accepted as they might be, 
study by Zoltan and Chapanis (1982) or* what professionals think abou: 
computers, over 500 certified public accountants, lawyers, pharmacists, 
and physicians in the Baltimore area filled out a 64-item ques t ior. ' a ire 
on their experiences with and attitudes toward electronic computi^ra* Si^: 
factors emerged from a factor analysis of the data. Factor 1^ the 
largest in terms of the variance accounted for. is a highly positive 
grouping of adjectives attesting to the competence and productivity of 
computers, such as efficient, precise, reliable, dependable, effective, 
and fast. Factor 11. the second largest in terms of the variance 
accounted for. is made up of highly negative adjectives: dehumanizing^ 
depersonalizing, impersonal, cold, and unforgiving. 

Still another factor in the 2ol tan-Chapanis study indicates 
discontent with computers in terms of their ease of use. The respondents 
thought that compv^ters are difficult and complicated and that computing 
languages are not simple to understand. These views are apparent in 
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their responses to such staten.er.ts as: -I would like a c puter to 
accept ordinary English staten^ents" and "I would like a con^puter to 
accept the Jargon of my profession." both of which they agreed with 
Strongly* 

The findings of that study are generally in agreen^ent with more 
infonnal reports in the popular press and other .edia about difficulties 
people have with computers and their use. Ir.deed, concerns about making 
:on.puters easy to use can have serious econon^c consequences that .^y 
have to be faced by more and more con^puter r.anuf acturers . For exat^ple, a 
small company in California was recently awarded a verdict for 
substantial n^onetary damages because of the inadequate perfor:Bance of a 
computer that the company had purchased (Bigelow, 1981). In rendering 
his opinion substantiating the award, the presiding judge said, "Ifs a 
particularly serious problem, it seems to .e . in the computer Industry, 
particularly in that part of the industry which makes computers for 
first-time users, and se^ks to expand the use of computers by . . . 
targeting as purchasers businesses that have never used computers before, 
who don't have any experience in them, and who don't know what the 
consequences are of a defect and a failure" (Bigelow. 1981:94). 

in Europe resistance to computerization has taken a somewhat 
different forn. than that in the United States, Television programs 
roughly equivalent to the American progran. 60^Minutes have been broadcast 
about the real and imagined evils of computers. Several 

countries-Austria, England, France, Germany, and Sweden among them-have 
prepared strict standards for the design of computer systems and have 
enacted federal laws restricting hours of work at computer terminals. 



127 



4 



r^p.':lniiOA^^ may soon be in effect In this counny. a-a 
cifficultv is that current standards and regulations about coF:7,i.t £ . 
sometimes b.Ased on skimpy and unreliable data and sometimee on no dat.. at 
all (Rupp, 1981). Whatever their origins, these events and trerids are 
svTnptoms of fairly widespread uneasiness and malaise about computers, 
their usefulness, and usability. No one denies that computers are here 
to stay. The important question is: -now can we best design them for 
effective human use?" This chapter describes some of the research needec 
to answer that question. 

Research needs are identified throughout the chapter. However 
desirable it might appear to assign specific priorities to each, ^ 
that it is difficult and risky to do -io for at least three reas^.. - 
First, computer hardware, software, and interface design features aia 
changing very rapidly (for a s-uinmary of the trends and progress in 
computer development see Branscomb. 1982)= So, for example, tn, 
increased availability of modularly arranged components for 
microcomputers for personal use. In the office and at school as well as 
new networking and communications features allow design improvements to 
be made quickly by trial and error. As Nickerson (1969) has pointed out, 
such trial-and-error design improvements can be made more quickly than 
they could be by careful laboratory research studies. 

Second, practical considerations are likely to be significant 
determinants of what research can be performed. Operational computer 
systems rarely can be disrupted for research purposes, and up-to-date 
hardware and software as well as appropriate groups of users are not 
always available. Under these circumstances It takes great ingenuity to 
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conduct huTDan factorG research on user-coisputer interactions that can 
produce useful, gf^neralizable results^ Constraints and opportunities are 
therefore more likely n assigned priorities to dictate what research 
is performed* 

Third, there is a definite need for good human factors research in 
all the areas we discuss, even with the caveat that technology is 
changing rapidly and good research is difficult to conduct. With these 
qualifications in mind, we do provide at certain places in this chapter, 
short suminarieE indicating those research needs that we feel have higher 
priorities than others. 

THE COJ^UTER SYSTEM 

Consputer systems and their environments have been diagrammed and modeled 
in various ways. Figure 5-1 illustrates elements that are important from 
a human factors standpoint: the user, the task, the hardware, the 
software, the procedures, and the work environment. Together they 
cluster fvTCund what is commonly called the user-computer interface — that 
invisible surface that binds the various elements together. Diagramming 
a cODpiJter system in this way is to a large extent artificial, because 
the various elements cannot really be considered in isolation. As will 
be apparent later on, there are interactions among all of them. The 
figure is -nierely a convenient way of structuring and organizing the 
subtopics of this chapter, which are described briefly below and treated 
in detail in subsequent sections* 

12.9 
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FIGURE 5-^1 Important Elements of Computer Systems 
Source: Adapted from Chapanis (1982)* 
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1-. The Users Beginning with the users is a natural starting point for 
any discussion of the human factors involved In coTpputer systems. 
Focusing on users implies what is sometimes referred to as user-oriented 
design^ rather than machine-oriented design. Perhaps the most important 
questions about users are "Who exactly are the users?" ''What are their 
characteristics?" and "How can user requirements be translated into 
cesigr. requireujents?" 

2. The Task The second element is the task or the job that the user has 
to do with the computer. The complexity of the job, the kinds of 
infcnsation the operator needs to perform the job, and the constraints 
under which jobs must be perfonrjed are all relevant considerations in the 
human factors design of computer systems. Task requirements are 
discussed in the section on users. 

-3. The Hardware Hardware means input devices, output displayy and 
signaling devices, and the work station that the computer operator has to 
use . 



4. The Software Software generally refers to the data bases, computer 
prDgran.5, and procedures available in a computer system. 

5. Procedures Procedures, manuals, and documentation are often Included 
under software* They are shown separately in Figure 5-1 because the 
problems associated with manuals and documentation are eomevhat different 
fro- thos^ associated with prograomiing languages, cOTSTsands, and menus. 
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•-_ - - - 'i':T.o;^ G<:neraliy sp^jaklng, cocput*jrs aro c<.opu::»'. 
.■.•r:ez:s a"e found in relatively benign work environmfjn'LS . So-'.c^ t c-- j.. s*- . 

■'^e leciLures of the work environiBent--excessive glare, noise, 
so-ezi-es dirt and vibration — hive to be considered in Lho de^ii'-c or ... 
Liser-conputer interface. Since standard human factors recoomjendatiou;: 
and good engineering practice are usually adequate guides for designing 
most work enviroriments in which computers are located, we do not cove: 
envi ronnental variables in this c'r.^^L^r ^ 

USERS AKD TASKS 

Computer users today are almost as varied as people in general. .Utncugh 
there have been a number of attempts to categorize or classify ccmpur'^r 
users Into various groups or along various dimensions, there is coo::./ ... 
generally accepted way of doing either* Computer tasks, by c:on;:rast, can 
be classified under the same headings as are used in task analyses. 
Proceeding from the more global to the mors detailed they are jobs, 
functions, tasks, and subtasks* According to Ramsey and Atwood (1979), 
most of the literature about computer tarks is at the job level- Some 
people think, however, that computer tasks cannot be classified in 
isolation, but that tasks interact with users and that the two must be 
treated together* Examples are: professional programmers designing 
sysLems^ professionals using application programs with command languages, 
occasional users using application programs with menus* In short, 
classifying computer users and tasks Is clearly in need of systematic 
work- and it Is treated more fully in the sections that follow* We rely 

1 :^ 2 



in Our discussion on the exemplary review of the literature on 
huaan-coisputer interaction by Ramsey and Atwood (1979), which .^a$ 
suDported by the Office of Naval Research. 

Users 

Attempts to classify users have followed one of several quite different 
approaches. The fi'st is to categorize users into more-cr-less distinct 
groups c~ the basis of their familiarity oi sophistication with 
cornputers. This way of classifying use^s has yielded a large collection 
of names. Examples, in alphabetical order^ are: casual users (Martin, 
1973), cospucer prof essior als (Barnard et al*, 1981), dedicated users 
(Martin, 1973), discretionary users (Bennett, 1979), experienced users 
(Shackel, 1981), familiar users (Ledgard et al., 1981) flrst-tinie users 
(Al^Awar et al., 1981), the general public (Shackel, 1981) ^ general users 
(Miller and Thomas. 1977), inexperienced users (Dzida et al., 1978), 
naive users (Thompsouj 1969), noncomputer specialists (Shackel, 1981), 
nonprogranmers (Martir,, 1973), occasional users (Hammond et al., 1980), 
prcgramnsers (Martin, 1973), regular users (Dzida et al., 1978), and 
untrained users (Martin, 1973) « 

Another vay of categorizing users has focused more on the nature of 
the user's job* This has produced such categories as: analysts (S. L- 
Sciith, 1981), clerical workers (Stewart, 1974), managers (Ec^son, 1974). 
operators (Smith, 1981), progranuLers (Martin, 1973), rugged operators 
(Martin, 1973), service personnel (Sisith, 1981), specialists (Stewart, 
1974), and technical users (Raissey and Atwood, 1979). 

1 



.,^:lc; <£i ciiif-.enL way of ciassifying users is iii L^rms ox uaaei in., 
personal ..harac te r is t ics . Thus, Ramsey and Atwood suggest: obtaiaini.> 
anr.^r users' abilities, acquired skills, general background (in^^aa^^^ 
loxz^^l education), sex, age, attitude measures, mechanical (perhaps also 
spatial) aptitudes, vocabulary test perfomance, recency and length of 
training periods, training scores, cognitive decision style, and geiieral 
intelligence = 

Another classification of uo. . h iharacteris tics would include data 
• n the following: 



1. Sensory capacities, e^g*, visual acuity 

2. Motor abilities, e.g., typi- , skills 

3. Ajithropometric dimensions, ^or hardware design 

4. Intellectual capacities, e*g*, general intelligence and special 
abilities in order to evaluate reading levels for infcra-i ti . 
presented 

5. Learned cognitive skills, including familiarity with the English 
language 

6- Mathematical and logical skills 

7. Experience with computers and proficiency in training 

8. Personality, e.g., attitudes toward computers 



Shneidermjn (1950), by contrast, classifies users only according to their 
semantic and syntactic knowledge about computers. This way of 
classifying users yields the simple matrix showna in Figure 5^2. 

The diversity or approaches that have been taken to this problem 
indicates that we need research to understand and identify which of many 
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langviage (JCL) 
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FIGURE 5^2 Classification of Users According to the Extent of Their 
SetTiantic and Svntactic Knovlprf&p 

~ y _ ._ C>~ 

Source: Adapted fronj Shnelderroan (1980)* 
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^~ -r-lMe user cr.u rac r e rl s r xc s are important for soft 
n l:.^ : c i or, , research Is needed to understand how to i> 
u.^'_r characteristics inro terms that can be used in 
Into specifications for designers of system scftu'are 
It is important to recognize that all users ^ whe 
seasoned -systems programmers or less experienced use 
as newer systems are developed and/or updated* For 
(1980) has suggested that we need to consider the ca 
computers as well as expert or naive users* Additio 
user behaviors could give us evidence of the functio 
e*g.^ the range of tasks users can perfDrro with a gi 
it takes a user to learn a system or i system update 
takes a user to perform a particular cask or job* W' 
kinds of errors users make when learning new systems 
errors are made and bow often they are made or repea' 
adapt to changes in system software (robustness) tha 
compatible,"* and how users rate subjectively the qui 
or product and the systems that perform their set of 
When we look at what is currently known about thi 
with the export user, it appears that the former is ^ 
problem solving and is very susceptible to task-struc 



*Upv3 rd compa tible means that commands and features l 
version of software are still available in a newer ve 
newer version may provide new commands or features th 
efficient for accor-plishing the same ends. 
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The expert systems programiner typically interacts with a computer as a 
routine ccgnitive skill and is somewhat immune to structural variations 
in the tasks performed (see Moran. 1981: Mayer, 1981)* A simple dialog 
in the software that Is computer^initiated and tutorial in nature is 
probably more appropriate for the occasional and naive user, but an 
abbreviated, user-initiated dialog appears to be more appropriate for the 
experienced user. It Is clear that we need to gather more data about 
problem-solving strategies and preferences across different types of 
tasks for different levels of users ^ 

Of particular concern is tht<t the research methods used in 8val'aatin£ 
user characteristics for hardware design have been used in studies 
evaluating user characteristics for software design- Xt is not known if 
these research isethods are approT>riate for evaluating softvare use or 
which methods will provide the most information to desi«mers« Moran 
(1^81 J has addressed this issue in t>art , 

Perhaps the two most r>rpRj:iriO ir*»c*»a-rc^ no^aHe ^ ♦'li'**^ o****** i' ^ 

some meaningful wav of classlfvlncr nr t*»oor -f y-f no ^^sfirc or^A f^emoi 04-4 
user characteristics into specific recommendations that can be used in 
the design of computer hardware, software, and documentation* 



Host computer and human factors specialists agree that a task taxonomy is 
needed and that system designers need a set of benchmark tasks to 
evaluate hardware/software development and changes. A task structure 
provides the rules cf :he game that determine the r^w^e of actions users 
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-ciii and cannot take (Moran 1981). Tasks can vary in several ways, rbey 
may J) fulfill different functions for the user, e.g., professional, 
educacional, or home hobby functions, (2) require different forms oi* 
Tanguage such as natural language, BASIC, COBOL, or,APL, and (3) be 
performed on different kinds of systems. 

In additi on, almost all systsm designers reccf^nlze that the user's 
interface with a computer system changes as tasks or jobs change. The 
user interface includes any pari, of the computer system that the user 
comes in contact with physically, perceptually, or conceptually, 
user's conceptual model of the system to be used to perform a given tEb>. 
is part of that interface. Thus, we also need research to undersc.^!^ : 

to discover a user*s conceptual modei(s) when he or she is interim 

with the computer. 

Models suggested by Moran (1981) Involve explicit information 
processes that spell out step--by^step the mental operations the user ir-^i w 
go through to complete the task application. These models need to be 
based on a psychological theory of users. One example of specific models 
that describe individual user differences in understanding calculator 
languages is described by Mayer and Bayman (1981). 

It would be helpful if a subset of tl.e taj.k taxonomy or benchmark 
tasks could in some way be integrated into the accounting systems of 
computers so that system designers could be provided with statistical 
data about tasks and users. These statistics on users should Include 
Information about the user type and systems used as well as errors in 
usage. One example of a keystrokes-level mocel for evaluating performance 
is described by Card et al. (1980). 
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Of priniary need are systematic studies of the conceptual models of 
users when they Interact with a variety of hardware and software systems 
to do specified sets of tasks, e.g., text editing, numerical problem 

solving, or querying data bases. Thesp Kfnrf-foe ay^r...^ ^ cb---^ -^i, -f,,i 

methodologies for producing results that can be directly applied to 
system design, or they should include new mpfbr.de f'^r 

interactions of user characteristics with tas^c renii-rr*»m*»Tit*^ An-*^^-^- 
^ --*='-*-o r — ^-i-^i^ t»»e uc vcxopmcut oj a meaningful task taxonomy that 

' — «^wwi. *^^*.v* T au*. a J. czuu v^^wgij X L J. ve elements for a set of four or five 

different representative tasks. 



COMPUTER HARDWARE 

Computer hardware cannot be designed i -fKnlation v^o^^^n^-^ *-h- 
hardware available on a comnuter teirm-fnfll r?r- t^T^rr-f «oe -r^ n«»r*- ♦■^^o I'-'^/^e --^ 
dialog and the kinds of command languages that can be implemented in the 
system. Ideally, decisions about important aspects of computer dialogs 
should precede decisions about terminal hardware* In practice, the 

reverse often occurs. While reco&nizina that ^h^6« •r^ter-a-*--' - 

and that they are imnortant In deislan r?-fe^iiee v.—"-- - 

aspects of computer hardware with only passing reference to their 
software implications. 
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Input Devices 

Designers of interactive computer systems can salect from a very large 
number of devices for inserting information Into computers. Table 5-1. 
modified from the work of Ramsey and Atwood (1979), lists 16 different 
kinds of input devices, comments on some of their features, and 
Identifies the principal references to studies of these devices. Since 
the situation has not changed materially since the Ramsey-Atwood 
report vas issued, its findings are still valid. 

By far most of the work on computer input devices has been done on 
keyboards: the literature As large and varied. Seibel's chapter in the 
van Cott and Kinkade (1972) handbook is a good starting point for anyone 
Interested in these problems. Ramsey 2nd Atvood reference a number of 
studies done after Seibel's chapter was written, and there is.^a fair 
amount of even newer work, e.g., Hirsch (1981) and Hornsby (1981), The 
available literature on keyboards is sufficient to answer most practical 
questions. This Is no longer an area urgently in need of extensive 
research. 

The situation with regard to alternative Input devices, such as light 
pens, touch panels, and hand printing, is different. Most of the work 
that has been done on these devices has compared two or more input 
devices in specific applications. There are not many studies of this 
kind in the literature, although Card et al, (1978) did evaluate the 
speed and accuracy of four devices for text selection. Research is 
needed that will lead to a set of recommendations ah^^z the kinds of 
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input devices that are best suited to general classes of tasks (e.;^., 

ip.put^ input of nuznerical datSj selection of comniands and op^rani"^?' 
iron: displays, discrete positional [graphical] input, and continuous 
positional [graphical] input) and perhaps to general classes of work 
e n V i r o nni e n t & • 

A much more serious concern is that there have been practicallj no 
studies of the optimal design of input devices, except for keyboards. 

That is* ijiven that a lieht nf^n is nArro-r rKan ir^v'^^«">-H 

applications* how exactly would oha rips'fsrn "Kc^o^ i-f£>^«- r-. -c-*- ' 

iob? Research is clearly n*»pdod OT^^■rTr.o^ a^^^o^ t--- 

input devices other than keyboards « 

Voice input to comouters dpAprvf- ; '^nec'^al ^T•^a a Vi^r>«-«ec 
does not involve a nbvslra^ me'^hari-t t-Via** ti^pr ^a*^-* t^»»^ « c-* 

and (2) sptseeh as a human output is distinctly diffirent froa the 

inovenients of finaer'^ bflnHc nr* fck^t- i-y\at- or-a «-A^,«-tv>^^ *-t- ^ -.i-^-.-^ , 

of uiost conventional cotDputer input devices. 

Speech has a number of chsr^tcteristics that theor^^tically sjake it an 

attractive candidate for coinputer inputs • It is fast, effective. 

versatile, flexible, and requires little effort. Moreover, almost 

evGryone knows how to talk, so that trailing is generally unnecessary. 

One of the principal reasons why speech input is not w*idely used. 

however, is that t<*f^hnnlnav h^c Koon aVil^^ ♦•^ -r^^^tT^^a 

# - - - - *- -fc^^ ..WW t^-wwM pA.wvj.sie wo wj. uiL opt^ecn 

recognition capabilities that even begin to approxlTuate thofe of huiDan 
listeners. Nonetheless, the state of the art is advancing rapidly. 
There are now some very good speech recognition devices available and 
the.1r capabilities are certain to increase greatly in the foreseeable 
future* 
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Although speech has some distinct advantages as a medium of 
communication, it is also easy to Identify applications in which speech 
input to computers would not be desirable. Some of these applications 
Involve certain kinds of users (for example, persons with speech 
iapedlments), others the task (for example, intricate mathematical and 
chemical formulae are not easily described orally), and still others the 
work environment (speech input is not very efficient in noisy 
environments). For more reliable guidance about applications in which 
the voice should or should not be used, the only source of help are 
recommendations comparing visual and auditory forms of presentation (see 
Table 5*2)* 

Table 5-2, and others like it in the hujian factors literature suffer 
from four major defects. -First, the recommendations are oriented more 
toward output devices rather than input devices — that is, they do not 
compare speech with other possible forms of data input. However 
attractive speech may appear as an input medium, some data are available 
suggesting that it is not necessarily the solution for all situations 
(see, for example, Braunstein and Anderson^ 1961). Second, 
recommendations such as those in Table 5-2 are not specifically oriented 
toward computer applications. Third- these comparisons are not 
sufficiently comprehensive to be of much use to computer designers. For 
example, none of these comparisons considers in detail user 
characteristics or the work environment in which computers are used. 
Some environments have rows and rows of computer terminals in close 
proximity. Imagine the babble that might result if 50 operators were 
inputting information by voice simultaneously into computers! Finally, 
existing comparisons of vision and audition provide Information that is 
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TABLE 5-2 Rec 
Presentation 



nidations for the Use of Auditory and Visual Forms of 



Us€ suditory prss^ntation if 2 



Jse visual presentation if: 



1* The isessage is simple* 

2* The ssessage is short « 

3. The message will not be 
referred to later* 

4« The message deals with events 
in tj 



:>* The message calls for iimsedi- 
ate action. 

6, The visual system of the person 

is overburdened, 

7, The deceiving location is too 

bright or dark-adaptation 
integrity is necessary* 

8, The person's job requires 

continual movement* 



1. 

2. 



The message is complex* 
The message is long. 



3« The message will be rereizc 
to later. 

4* The message deals vi;' 
location in space ^ 

5* The message does not call 
for immediate action* 

6* The auditory systexr ^ tre 
person is overbur jr!iie i * 

7. The receiving 1 ^•cation is 
too noisy. 



8- The person* E job allows for 
a stationary position* 



Source: Deatherage (1972). 



148 



25 

too vague to be of any practical use to a computer designer. For 
exaxDnle, how is a designer to decide whether a iriessage is simple or 
complex? 

What we clearly need is a detailed^ comprehensive, and quantitative 
set of guidelines about the precise conditions under which speech input 
to computers is and Is not desirable* These guidelines should consider 
the user^ the task^ and the work environment in which computers are 
located - 

Although some very good speech recognition iiiachines are available, 
they have some important limitations. First, they all are word 
recognition devices, that is, they do not recognize continuous speech* 
Second, they are capable of responding only to vocabularies of restricted 
size* Third, they are user-dependent, that is, they must be programmed 
to learn to recognize words spoken by a particular person and will 
generally respond aqcurately only to that person's voice* Speech 
recognition machines that can respond to connected speech or that are 
speaker-independent are well beyond the current state of technology* 

Despite these important limitations, speech input to conputers can be 
successful and useful. There Is not, however, a good base of research 
findings on the conditions under which speech recognition machines can be 
used effectively .even with their limitations. For example, how such 
useful work can be done with vocabularies of various sizes? How 
effectively can people be trained to leave pauses between words in 
connected speech so that individual words can be recognized? How 
effortful is it to speak while deDlberately leaving pauses between 
words? If vocabularies of restricted size must be used, how effectively 
can one construct complex inputs with the available words? What rules of 
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rannnar and syntax must be observed if one is restricted to 
vocabulary? What should that vocabulary be? The coadition 
speech recognition devices can be used most effectively is 
unexplored area of research that should be vigorously pursu 
example of research in the use of voice input to operate a 
computer network has been conducted at the Navy Postgraduat 
Poock (1980). 

Output Devices 

Although teletypewriters and alphanunierlc cathode ray tube ( 
are the most common forms of output devices used in computes 
there are numerous other possibilities: plasma displays; li 
diodes (LED) and liquid crystal displays? tactile displays; 
displays, including synthetic speech; graphical displays; la 
and even psychophysiological output devices* The state of t 
these various output devices is summarized in Table 5-3, whi 
on Ramsey and Atwood (1979) . 

CRT Displays 

Enough research has been done on CRT displays to support gxih 
their design (Galitz. 1981; Shurtleff, 1980). Although the i 
available do not answer all the questions designers may have 
3 substantial number of th#»m. Most of their recommendations 
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TABix 5*3 Coritluuev 



User dlspUya 



Psychophyslo- 

loRlcal 

diinlaya 



UrR€=scr€£ri 
dlipUya 



„.du.. ceapMheniion), CuldiiiBel art Bot knova to ssist 
but could b€ constructed with iddition.l lutTey of typogfipulc 
r.».rrh lltwiturt. iii. of line pflnttrs for -pisudo.r.phlc 
jl-pi.y. 18 cosson but little dlicuiiw in the literature. ^ 
Fieudographlci is m iafsp«nslw wsy to coflwy iiiplt grap-uca* 
infor-^tion and ihould pfubibly b£ used aors widely in batch 
ippllcatioQs« 

i^monabla huua factori Ruldsiir.ss with rsspsct to tisuil 
.:operti« h-« been propoied, but iheaa dliplays ara not 
widely used. 

I 

»,.ujj„g}, soa- tactile dliplayi have beer, proposed cr even 
developed, little haass factors research has been dona othtr 
than that concerned with prosthetics. 
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supported by research data, and those that are not seem reasonable. The 
two most important unresolved questions concern the size of displays and 
the use of colored displays. 

With reaard to size- 5;hin-Mef^ ''lOgnN h^^^ ^^^rr^^^^.^ « 
questions of legibility as related to display size, but he has nothing to 
say about the more important question of how much inforisation can be 
presented on screens of various sizes* Military applications of computer 
displays, for example, in cockpits, must be small by necessity. Knw 
small can they he and still be legible? How can inforiaaticn best be 
presented on small HlfinlavKv The po^^'^^'c** r»'»"n>^^ ••u^— 

People must vipw the KflTne '^iS'^la'" Tn t-ho*- ^oo^ *-v.« <^^-i A*«'-n«' ^ 

j^^L >«^wp^ii>B i^t;* LXK^m i^Aja xnx oruici u J. uu ucbt DC presenteu 

^a-^c E> e> . ^ftftCfoc^ »A c tivt qucoux^^no rcx<n-j.ug eij.upj.y to the 
legibility of the information presented on displays of various size: such 
questions can easily be resolved on the basis of available data. What Is 
needed is research on the interactions between display size and the 
amount of information that can be most effectively presented. 

Questions on the use of color on CRT displays is also still 
essentially unresolved. The advantages of color coding for 
identification purposes are. of course, well documented, but th^ 

• - * w ~ ^ ^ 

long-term effects of work^x^g with colored CRT disnlavs for data Pntrv 

~- - - ^ — ^ ^ 

inquiry, or interactive dialog are not known. Although many people seem 
to like colored displays, others find them annoying and garish. The 
scanty research evidence available seems to show that colored CRT 
displays produce no substantial perf CL.^acce benefits* More research may 
enable designers to make inforuied decisions about the possible benefits 
of color on CRTs versus their cost and o^her disadvantages. 
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Alternatives to CRT Displays 

Very little hmaan factors research has been done on displays other tb^n 
CRTs. Of particular interest are synthetic speech displays. 
Computer-generated speech is now available in a variety of devices, and 
the quality of the speech in sorrc^ of these devices is quite good « The 
situations in which computer-generated speech is a viable alternative to 
visual displays^ however, are not known , Basic research paralleling 
that on speech input is needed to produce defensible recommendations 
about applications in which speech output can or should be used* 

Workplace Design 

Computer displays and input devices are generally assembled into work 
stations consisting of termiuals. consoles, desks, and chairs. There is^ 
of course* a very large and useful literature on the physical layout of 
workplaces (see, for example* Van Cott and Kinkade, 1972)^ but there is 
very little empirical research on work station design specifically for 
computer-related tasks and settings. The importance of these problems is 
highlighted by a great deal of literature, mostly from Europe, about 
complaints from workers using CRT devices (see, for example. Grand jean 
and Vigliani, 1980). 

Similar complaints from a consortium of labor unions in the United 
States were received by the National Institute of Oc.'-upational Safety and 
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Health (NIOSH) in 1979. The general nature of these complaints was that 
enployees using CRT terminals experienced a variety of symptoms including 
headaches, general malaise, eyestrain, and other visual and 
musculoskeletal problems. In response to these complaints NIOSH 
conducted an extensive investigation of computer work stations in three 
companies in the San Francisco Bay area (Murray et al=, 1981). The study 
consisted of four phases: (1) radiation measurements, (2) industrial 
hygiene sampling, (3) a survey of health complaints and psychological 
ffiood states, and (4) ergonomics and human factors measurements. 

Although radiation from CRTs had long been suspected as a potential 
health hazard, the NIOSH study seems to have conclusively ruled it out. 
X-ray, ultraviolet, and radio-frequency radiation in all sites and at all 
work stations tested was either not detectable or was well below 
acceptable occupational levels. Similar negative conclusions were 
reached about the chemical environment. Hydrocarbon, carbon monoxide, 
acetic acid, and formaldehyde levels in and around work stations were not 
appreciably different from what one would find In an ordinary living 
environment. 

The results of the survey of health complaints were quite different, 
however. They show that operators of visual display terminals (VDT) 
experienced a greater number of health complaints, particularly related 
to emotional and gastrointestinal problems, than did comparable operators 
who did not work with VDTs . These findings, according to the NIOSH 
report, demonstrate a level of emotional distress for the VDT operators 
that could have potential long-term health consequences. The NIOSH study 
concludes, however, that it is quite likely that the emotional distress 
shown by the VDT operators is more related to the type of work activity 
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Chan to the use of VDTs per se. With the growing number of VDTs in our 
society, it is clearly of considerable importance to establish how Each 
of worker complaints can be traced to VDTs and how much to other factors 
(Ketchel, 1981: M. J. Smith, 1981). This is a research question that 
urgently needs to be Investigated* 

The hnoSH report has more to say about the ergonomic and human 
factors aspects of the computer workplace than about any other aspect of 
computer work- Keyboard heights, t&ble and chair designs, viewing 
distances and viewing angles, copy holders, and other aspects of work 
station design all come in for criticism. Computer work stations Xn 
America appear to be as poorly designed as those in Europe (see Grandjeao 
and Vigllanl, 1980; Brown et al*, 19^^2), forcing operators to adopt 
strained postures and to contend with glare and generally substandard 
viewing conditions (Ketchel, 1981) « Although basic data for good work 
station design are available, they need to be assembled In a good set of 
guidelines specifically oriented toward such design. This also appears 
to be an urgent research need* 



General Problems 

Three general problems relating to computer hardware have received almost 
no attention: (1) the design of transportable terminals and data, (2) 
the design of robust computer systems for military purposes, and' (3) the 
design of computer terminals for use in unusual or exotic environments. 
for eAdoplej in moving vehicles or under water. 
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Spectacular advances in microelectronics have made it possible to 
package enormous computing power into small packages. The full potential 
of this miniaturization has not yet been realized oi explored. We need 

human ^ACtrtrs -rocoaT^y-K 1 j4 -t riCT •■ -2 ««j . ^ 

~ — -1.*- ±^a\A^Li^ Z\j Cue uctoigu and use of transportable 

p o p'-w ouupuc ucvxi^es ana data in the form or 

cassettes* 

Most computer systems are designed for use in benign environments. 
As the use of computers becomes more common in the military services, 
data will be urgently needed on how to design them for the rough 
treatment they are almost certiin to receive under operational conditions* 

Vibration, high-g forces, immersion in water, and perhaps other 
environmental conditions affect machines as well as their operators. 
Certain input devices, for example, light pens or even keyboards, may be 
difficult or impossible to use when the computer and the operator are 
subjected to excessive movemer^C;, vibration, or g^forces- We have 
essentially no infonnatlon about the usability of computers ov the design 
of computers for use under such conditions. Although this may not be an 
Immediate problem, it is certain to become increasingly important as 
computers are integrated into complex systems for use in harsh exotic, 
or unusual environments* 



CO>ffUTER SOFTWARE 

Software has many different meanings to computer scientists and computer 
analysts who develop or use computer programs that Include command 
languages, dialog systems, and specialized applications systems with data 
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bases. Software may have originally been synonymous with computer 
programs, but in general software now consists of "the operational 
requir? i^enis for a system, its specifications, design, and programs, all 
its user manuals and guides, and its mainLenance documentation** (Mills, 
1980:417). 

Research in human factors in software has evaluated the 
huniau-coiriput er interface with command languages programming languages, 
dialog systems, and feedback and error management. Frequently the human 
factors studies have emphasized ease of use and ease of learning as well 
as efficiency of completing the problem-solving tasks on the computer. 
The recent experimental and observational studies were summarized ia the 
special issue ou human factors in C ompu c i n g Surveys (1981). the IBM 
Systems Jour nal (1981). and in articles in Human Factors , the 
Id teraational Journal of Man Machine Studies ^ and Ergonomics . In 
audition, there are exemplary technical reports, such as Williges and 
yilllges (1981), Ledgard et al. (1981), Shneiderman (1980), and the 
proceedings of the Conference on Human Factors in Computv-^r Systems 
(Institute for Computer Sciences and Technology, 1962). The more popular 
trade magazines, e.g.. the April 1982 issue of BYTE > also feature 
articles on human factors in software design. Many authors express the 
need for additional careful research studies in software design and 
critici2e many current results as incomplete and inconsistent due to poor 
me thodology J use of subject populations llaiit^'d to particular types of 
users (e,g*, college students), inadequate experimental designs," and 
misuse or poor use of statistics* 

Selected useful guidelines for software designers are found in Engle 
and Granda (1975) and the recent reports by Willigec r.nd Williges (1981) 
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ar.d Ehrenreich (1981). Although there exist guidelines as well as 
selected research studies in human factors Issues In software, 
considerable research needs to be done In order to provide Infor^tion of 
use to system designers of software. 

The research efforts needed in hioaan factors in software design can 
be divided into two areas: (1) methodc logical studies and (2) 
substantive studies of software design f.^.atures for the end user. The 
two areas are not always independent, and souse research studies require 
attention to both. In either case we are concerned about human factors 
research in software systems with which end V'3ers interact or interface, 
not about research in programming language :..isign per se; thj.s is usually 
the concern of the computer programmer or systems aaslyst. 

In the methodological area, research is needed on how to develop a 
suitable simulation capability for the design of dialog and interface 
systems. We need to understand how to evaluate present software systems 
as well as how to mock up new systems for testing and evaluation with end 
users. The choice of dependent variables In evaluating software is not 
clear. We know little about how to collect user star ■sties on the ease 
of learning of new software, how to record errors and complex 
response-time metrics from end users in time-sharing systems, and how to 
measure user satisfaction. Research Is needed on what components of 
usability are most important for different kinds of users and 
applications (see Shackel, 1981). 

One of the problems in this area is that we don't know how to do 
research on these topics. There is no agreed-upon set of empirical 
methodologies for conducting research studies about software issues. The 
studies that have been done are frequently context-specific and/or about 
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one or two software features and are difficult to generalize and 
Integrate with other data in the area. Examples include evaluations of a 
given comnjand asking users to translate the abbreviated form into 
English, effects of modifications of conditional nesting structures in 
FORTRAN, user efficiency of indentations to locate single bugs in PASCAL, 
and iLodif ica ticns in a language used in teaching at the University of 
Toronto. A research program undertaken by a multidisciplinary group at 
Virginia Polytechnic Institute and State University by Williges and 
Ehr:'ch sponsored by the Office of Naval Research [ human-coniputer 
interaction and decision behavior, NR SRO-lOl] is attempting to develop 
principles of effective humane-computer interaction, including 
establishment of a user's model of command languages. This research is 
interdisciplinary and programmatic in nature. Another set of 
methodological studies is needed to discover how to develop guidelines 
and what kinds of guidelines for software characteristics are most useful 
for system designers and engineers: for example. Smith has described his 
ideas and progres:. in this area in the proceedings of the Conference on 
Human Factors in Computing Systems (Institute for Computer Sciences and 
Technology, 1982). 

In a substantive area, research is needed to understand the control 
of users' input accuracy through "clever** or "novel" feedback during 
actual user experiences as well as wh_at the "format structures*' should be 
for providing feedback on errors that users make = Data rjeeds to be 
collected on how best to provide effective error correction features, 
help messages, and what range of default procedure?^- should be provided to 
aid user efficiency. We need research to evaluate how important feedback 
and system response time are for improving user effirtency or ease of 
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use. There is a need for methodology and quantification of user ease and 
efficiency- At pr^se-it, studies evaluate different types of coimnands in 
a laboratory rather than in real-use settings, and it is not clear that 
the tnost effective connnands in the laboratory are applicable in applied 
systetu uses. Ve need informs ^I'^n on what length of commands (one. two. 
or three -words) or how many ^enter only one and wait for system esponse 
or enter six at once) are preferred by casual users rather than expert 
software programmers. 

variety of studies are needed in order to evaluate how best to 
develop natural language dialog systems and In particular what kinds of 
language-based models of human communication are most appropriate for 
commands in operating systems, editing systems, knowledge-based systems 
anc query systems for human computer interactions (e.g., Reisner, 1981). 

Additional reseav-h is needed to understand bow tc develop 
knowledge-based systems for a variety of users. Knowledge-based systems 
are developed by a formulation of the application problem, designing and 
constructing the knowledge base of expertise, developing schemes of 
inference, search, or problem solving, winning the confidence of experts, 
and evaluating the programs for production versions- Examples of 
knowledge-based systems, frequently referred to as expert syHL^.ass, 
include assisting users in such tasks as: (1) deducing molecuUr 
structures fro- the outpi^t of mass spectrometers. (2) advising when and 
where to drill for ore, and (3) diagnosing blood infections . It should . 
be noted that there are three different kinds of end users of these 
systems, only the first of which is a user in a conventional information 
retrieval system: (1) in getting answers to problems, the user as 
client, (2) in Improving the system's knowledge, the user as a tutor, and 
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(3) in harvesting the knowledge base ^ the user as pupil- A sunmiary of 
recent research related to knowledge-based or expert systems can be found 
in C* Snjlth (1980)^ Some of the major features of these systems > 
Including the schemes of inference or problem-solving approaches used in 
defining structures for r.he knowledge bases, are reviewed by Feigenba'an] 
(1978) . 

A recently developed specialty is software associated with special 
grapnlcE displays* At present the development of both hardware and 
software for graphics use are at the gadget stage* We need to know how 
to design software modules for graphics use, what modules are best for 
various graphics features in addition to points , lines, and circles, and 
how to mix keyboard and pen inputs in ways other than up and down a ows 
and drawing pad devices* Host graphic software has hierarchical levels 
for command use; it is unknown if differeat levels are needed or how many 
are needed and which comsiands are best to use at each level* Also, the 
best ways for interacting among the hierarchically ordered levels of 
commands for draw and edit and the method for terminating are unknown. 
We need more information about what icons, menus, and special symbols 
should be used in creating graphics* Methods have been developed for 
partitioning a display screen into multiple, sometimes overlapping 
windows, each monitoring an independent process* There has been very 
little research on how best to make use of this kind of capability- We 
know little about how to use color effectively for different kinds of 
graphics displays and applications. 

Several of the above research recommendatiors have been recognized by 
Hi ran (1981), who also suggests that further research is needed to 
understand users' conceptual models in interacting with a variety of 
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software systems. In addition, Thonias and Carro 
(1981) have emphasized that the areas of most ne 
human-to-computer conmiunl' ition process, includi 
advantages and disadvantages of natural language 
different tacks. Computers have become more a p 
today, and we need to study what impact the new 
on organizations and their structures as well as 
making of the new management inf orniation systems 

Ae a final point, it should be noted that wa 
interaction between hardware and software design 
developments such as voice input and video disks 
incorporated into all types of computer systems. 

Important research that should be done invol 
analysis of new methodologies for conducting sof 
second, users' conceptual models of software sys 
language systems for a variety of tasks. Also, 
to develop and evaluate additional knowledge--bas 
client, tutor, and/or pupil. ..l/.rj needed are st 
understand what software features would facilita 
graphics in different tasks. 

DOCUMENTATION 

Documentation was once defined as printed matter 
explains how a system of some kind works cr shou 
documentation was necessarily separate from the 
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itself was a thing of print on paper = In the context of the coisiputer, 

however, documentation can be part and parcel of the systeui it describes 

or exnlains. Recent experience indicates that on-line clccuiiientation has 

many advantages over print-on-paper documentation* -It cannot get lost or 

separated from the system* Inasiuuch as the user is working with the 

comnuter. the computer can monitor wh_at the user is doing and help find 

the Darts of the documentation that are pertinent to the user's current 

activity and current quandary s When the user thinks he or she 

understands what to do the computer can help do it — and may be able to 

*. — r»««*- * n f n ■( waxr t-Via f w*fll not cause much trouble if the 

,.^«_T^ j^^of-or^/ii r^p •( c fauT^v= The noBsibilities are obviously 

revolutions "y« Because on""line documentation is relatively new, however p 

not much Is known about how to design and implement it effectively. 

Clearly the first priority for research in documentation is to explore, 

„ 1..^-.- -I «*7Nr torhri'Jnu^K of on— line documentation* 

evaxuai^e* aUia jLuipiw.w i.*-*,-* 1 — *- - 

On— line documentation within the systeto is not the answer. -to all 
needs for documentation > of course* Some computer systems (such as 
V n A p p < na e^vctpTnc «T^d flnfnmBtlc orocess— control systems} arc 

uaC V-»»—p*. «^C Ug '-'J — U tf U tr - 

norin t erac t ive , and others (such as many avionics systemSy do not fcave 
enouj^h memory or storage to make on— line documentation feasible = 

Documentation for such systems ie>- L»y onu i-iOi. ^^^j - — - 

There is still need, therefore > for improved external documentation, 

jt^ a t- •{ r^a*- *^ accr*! -fated with the system but not in it. Wright 

fl9Sl) has several useful suggestioiis for documenLEtlon designers, 
including suggested aids that take the form of heurlsfics for analyzing 
the user's interaction with the text. Her suggestions also consider 
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types of users and the user's (reader's) purpose rather than the producer 
designer's (writer's) purpose as a classification for documents^ 
Of course, external documentation need not necessarily be 
:nt-on~paper documentation. It is an Interesting Idea to associate a 
"documentation computer- with the system to ^hf.ch the documentation 
perta-tns. In some instances, the documentation compcttr might be a small 
machine, even a portable one. taking the vl'^c^^ of a ew manuals; other 
instances — those that have veritable libi ar., . l ! /^.>cumentation — might 
require a documentation computer system of £ : ricant sire. In a^ 
experimental system on an aircraft carrier, for example, liie coup-jtc: 
system that handles documentation is a network of about 30 PEPQs* that 
are 16-bltj chip--bas.d "personal" computers of substantial capability. 

Documeatar ion as Fart of an Overall System 

The aircraft carrier project introduces a concept that will no doubt be 
very important in the future: Documentation and what users do with it 
are parts of a larger tysteiii- If the use of documentation leads to the 
discovery of a defective part , inventory must be checked and ordering may 
have to be done. If the use of documentation leads to isolation of a 
soft* are bug^ software maintenance jork must be done* It would be 
convenient and would foster efficiency if the same system that handled 
documentation also handled inventory and software maintenance. To 
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improve the overall effectiveness of documentation, research is needed on 
the interactions rf docunjentatlon with other parts of the overall task 
Support S3'*^te!ii« 

Computer- Based Versus Print-cn-Paper BocuEentaticn 

The- c, cussion thus far has focused on computer-based docutnent ^n ^ even 
when ihe system being documented is not itself an interactive computer 
system. That choice reflects the judgment that rese-rch in 
computer-based documentation is more likely to make a iDalor payoff than 
ongoing research in print-on-paper documentation- The latter research 
has led to many improvements and the total effect has been significant, 
but, Insofar as conventional documentation is concerned, diminishing 
returns have set iD= Computer-based documentation, by contrast, with the 
capability of the computer^ offers hope of a very major advance = While 
computer-based documentation is not a new concept by any means, it has 
just recentJy begun to be studied systematlrally . The "help systeins" and 
the "tutorials" of the 196us and 1970s were written without the benefit 
of research of the kind that was devoted, for example, to programming 
languages. As a result, it has been said, the help systems needed help 
systems and the tutorials needed tutors. Our conclusion is that now is 
the time to make a strong research attack on computer-based 
documentation. Including self-instructional programs, coherent 
system-wide help systems, dccumentatior. keyed to the behavior of programs 
( .'O that an error calls forth ar. explanation of what went wrong), and 
programming languages that write programs to explain themselves. 



Capturing the Intent of the Creators of the System 

As suggested earlier, docuisentation miist viewed as a part of the 
overall system that nteracts vith other parts of the overall system* 
The time dimen.- the history—of the overall system is a very 

important base of che interaction. Most systems are developed through 
efforts to improve earlier systeuis, and those that do not are developed 
from some kind of design activity In the minds of system designers. 
(Programs are systems, of rr..---.^ r.o the same can be said of programs). 
The Intentions of the Improvers and designers are crucially important to 
understanding what the systems do, how they work, and how they should be 
used — but intent, as tend not to be captured in the plans and designs, A 
computer program, for example, usually tells bow to do something, not 
what It is that is being done, and it Is very difficult to reconstruct 
the programnier-£ intentions from the program. Research on this topic may 
or may not improve the situation, but it clear that the sifiatlon needs 
to be improved^ A broad view of documentar. 1 is important^* The right 
approii-n may be to create computer-based uci.ign and upgrading 
-etasystemsj within which Improvers and designers would work under 
constant monitoring, with as much emphasis on recording Intentions and 
goals as on devising the means for achieving them. Note that this 
notion. If not developed with sensitivity to privacy issues, could lead 
to serious ethical problems* 



Dynamic Graphics and Documentation 



Although documentatioTi vas, in earlier days, primarily print on paper, 
some docunsentation has been available In other media, such as recorded 
speech and movies. The latter offered^ at considerable cost, the 
advantages of kinematic graphics and moving gray-scale and color 
pictures. The computer promises to reduce the cost of preparing 
kitiematic graphics by having a single, static program create dynamic 
TBultldimensicnal patterns that develop over time, Th- video disk 
promises rc reduce the cost of storing and playing back all kinds of 
information, e.^pecially pictorial 1 nf ortnation . Together the computer and 
the video di. \ii^y open up . new era f.-^r dynamic graphic documenration. 
At present the cc'-ruter cau select and v.^-y^ent in a few milliseconds any 
one of the approximately ^' pr>/j p/..^,..-^. p video disk.- It^-can run off 
sequences of continuous frames as a movie or skip around under progiam 
control and show fast slice sequences* What it selects can be 
cordi^-f -ined, of course, by the responses of the viewer or viewers. These 
capabilities present an exciting opportunity to explore and develop new 
approaches to documentation • 

Another exciting opportunity is being studied under the rubric of 
program visualization. The computer is capable, of course, of displaying 
representations of its own internal operation. It can present sequences 
of symbols representing the program that is being executed ar.il • p da^s 
on which the program is operating. Alternatively, it can present graphs, 
diagrams, and pictures to tell the person at the console what tho p /ogrr-m 
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should be doing and what it ii in fact doing. This latter approach to 
docuaentation, which requires sophisticated graphi oisplay not videly 
available in the past, is now economically as well as technically 
feasible. The hope is that iconic displays will prove superior to 
syTDbolic displays in presenting the broad picture of the behavior of 
coiiputer programs and systems and in helping people deal with their 
intrinsic complexity. With the iconic approach^ it may be possible to 
provide something analogous to a zoom lens, through which one would be 
able to monitor and control the broad picture as long as everything 
proceeds according to plan, then focus cn the offending details as soon 
as trouble arises. 

Docunieu t a t ion in the Form of Knowledge Bases 

Conventional documentation takes the forms of natural language text, 
diagrams, sketches, pictures, and tables of data; it Is designed 
exclusively to be read by eye- New forms of documentation are becoming 
essential: pointer structures, semantic networks, procedural networks, 
and production rules, documentation designed to be Interpreted by 
computer programs. Such documentation will probably be used first in 
interactive computer systems to help end users or programmers and 
maintenance workers, but in due course it will be used also in fully 
automatic system.: sophisticated enough to read their own documentation 
and restructure themselves to overcome difficulties and maximize 
perf or ...iuce . Some work has already been done on such documentation in 
the field of artificial Intelligence; much more needs to be done* It is 
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essential to couple research on documentation closely with other research 

pertinent to the systems in which it will be used — for example, with work 

on inter-^ctl ve lutrorial systems for end users, interactive maintenance 

systems, and . ^c>otic maufacturing systems* 

« 

r 

Computer Systems to Facilitate Conventional Documentation 

The foregoing emphasis on computer-based docutrsen tation expresses our 
conviction that it is the high-payoff area within the docunientation 
field, but it should not be taken to imply that conventional 
documentation is dead. We think that two maiu foci have the greatest 
potential payoff for research in conventional documentation: (1) 
understanding the target group of people that ::he documeata tion is 
intended to help and the tasks in which they will be engaged when they 
use the documentation and (2) using computer systems j with good editors, 
formatters, and composers to facilitate creation and production of 
conventional documentation. 

The theme of understanding the users is developed elsewhere in this 
chapte^^. Q,rr.Bt advances have been made in the last few years in the 
design of cotaputer-based systems for creating and producing conventional 
docuipents, and research in that area has much new technology to work on. 
Indeed^ research is needed to develop the capability to saakc the new 
editors, fonsatters, and composers easy to use in order to facilitate the 
preparation of documentation t. :Jt will make them and other systems easy 
to use. Kruesi, for example, supported by the Office of Naval Research 
(hTR 196-160), is investigating the relationship between the types of 
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documentation provided to progranmier s and their performance on a wide 
variety of software-related tasks. 

In summary, research should be emphasized in several sreas pertinent 
to documentation: (1) techniques of on-line documentation, (2) 
Interactions and information flows between document subsystems and other 
subsystems, (3) efforts to capture the intent of designers and upgraders 
of systems, (4) dynamic graphics and the video disk, (5) dynamic graphics 
aod program visualization, (6) knowledge bases, (7) understanding the 
uses and users of documentation, and (8) computer-based systems f r the 
development of conventional documentation, of these suggestions tvo 
primary research needs are to know how and when to use display 
documentation witu graphics and what program visualization techniques are 
most helpful to users. 



SUMMARY AND CONCLUSIONS 



The primary research recommendatons in the areas of users, tasks, 
hardware, software, and documentation include a major emphasis on 
developing new methodologies to evaluate what is meant by ease of use in 
human-computer Interaction, Does ease of use mean the extent to which it 
is easy to learn to use a computer; does it imply good design of hardware 
and software for a variety of naive, casual, and professional users; does 
it mean that any task can be done quickly and without errors; does it 
euccmpast. a component of judged satisfaction about use; or does it mean 
all cf these? 
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We need to knov what user characteristics are important determinants 
of successful hunian-cOEputer interaction for s specified set o.f tasks, 
such as data base inquiries, coEsputat ion and accounting problems, and 
editor or word processing functions. In the area of hardware design, 
more research is needed to evaluate alternatives to keyboard input 
(including voice input), uses of color in displays, the best sizes of 
displays, and alternatives to CRT displays. Studies in evaluating 
software are barely beginning to provide data for design u: - We don't 
yet know how to conduct systematic research studies in software design ^ 
what independent variables are most iuipcrtantj and what dependent 
variables of hurDan-computer interaction shc^ild be recorded. We don*t 
have data to support the design of a simulation facility to effectively 
evaluate commands in operating' ^vstems, editing systems, knowledge-based 
systems, and query systems. We need to understand users' conceptual 
models in interacting with specific software systems, and we need more 
information about the advantages and disadvantages of natural ..language 
software systems. Documentation may well become part of the available 
software for users: when and how to display documentation is an important 
area for research. Research is ^.eeded on how best to use graphiei5 and 
special know' -^dge bases to facilitate uses of documentation either on 
line or in n:anu:iTs . Current documentation is designer— oriented rather 
than user-orl ented J and the perspectives should be changed so that 
documentation is used more effectively^ 

Although the research needs outlined are nu^.^rous^ a major emphasis 
in this chapter is on systematic studies that include all four 
substantive variables — user and task characterisiics ^ hardware^ software, 
and docuTTientation — and the interaction of these coniponents with a 
cle;*r-cut set of studies to define ease of use. 
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POPULATION GROUP DIFFi 



Many areas of research in human factors have < 
fit the average person. In those studies, inc 
traditionally have been treated as little more 
Thus few data are available in niany areas of I 
interaction of different systems with variable 
age levels. Attempts to classify^ describe^ f 
individual and group differences extend to the 
history*. Some of the earliest decipherable sa 
references to the physical and mental differec 
serf^: ^vd nob](»men^ slaves and masters, anc ba 
persons. was not until the nineteenth cent 

study of individual and group differences assu 
rigorous qualities of Fc-«entific Investigation 
Francis Galton (1822-1^11) to aescribe the nat 
differences are the foundrcions of vbat is Fom 
differential psychology* 



The principal authors of this charier tre Irvii 
Chapanis . 



Slhce 0.1^, investigations of Individual and g'roup differences 
c...l.d'out by psychologists. anth.opoIogl.ts. and sociologist, nu.ber In 
th. Hundreds of thousands, there Is . psychological journal, I1,^_Jou^ 
°£ j:ross-eulrnral _jsych^ entirely devoted to studies of this kind, 
one of the .ost Important applications of this work In psychology has 
been the development of a .ultlmllllon dollar testing Ix^dustry. 
Psychologists h.v. devised hundreds of tests of ability, achievement, 
skills, knowledge, and personality (Euros 1978) th.t .re used routinely" 
fo. classifying and selecting e^ij^loyees for thousands of Jobs and 
. occupations.^ 

one Of the .o.t ambitious and thorough attempts to relate Individual 
characteristics of Workers to job requirements Is the l^^lon.^y pf 




gives profiles of th. educational, ptltude. Interest/ physlc.l/and ' 
temperament characteristics required of a worker to achieve a^^rage 
successful job performance in thOusands'of occupations; The llltary 
services h.v. tried .to do so..ethlng similar on a ^or. inodest scale. In • 
the preparation of personnel requirements' data, the Air Forc^Jesl^^ 

(Air Force Systems Connn.nd. 1969) specif leS .tha t tasks should b. 
rated along six dimensions: ambient enviroranent . equipment 
characteristics, mental demands, physical de..nd. . ha.ard exposure, .nd 
task crltlcallty. Figure 6-1 shows the three leV.ls of mental demands 
ithat may be re^x^lred of people by various duties and tasks. 



^snf'm^^mflS:^^''^!°^''*M T^''^' diagnosing .nd 
activities. "-tnesses. but oar coilKrn here Is with job-related 
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fiGDE 1 requires little or no fortnal trainings just a baStc: intfbdu^tlon 
to_the task^ abtii^ relatively simple vtftteiS or oral 

^^^■^*«i±p«^ Tl^ since only elementary decisions involved; 

little concentration; little or no recall of relent Jcnpvledge for 
decisions or inference; only precise determination^ such as Gd/NG)~Gb 
UP/DOWN, MORE/tESS, YES/NO. ALL/NONE, CORRECT/INCORBECgk^fct^p. 



tODE 2 requires moderate technical knowledge and train^tn^-^ gome ability 
to adjust to changing situations; occasional exercise af. Judgment 
involving use of technical knowledge; ability td^ upderstmui and use 
technical manuals; some Initiative and Ingenuity required; occasional 
recall of relevant knowledge and experience of the pract^c*! type for 
decisions or inferences; decisions involving sbmeVhat detail^ procedqre 
or' measurements, as in assembling, disassembling, installing, feSoving^^ 
inspecting, testing, operating, adjusting^ computing, monitoring, 
servicing, etc; 

eODE 3 requires a high degree of complex and varied technical kflowiedj^e, 
with considerable formal and informal training; a high degree of 
continuous ^cbncehtration, with attention to advanced and Involved 
elements of the task; continuous exercise of a high degree of judgtnen^i 
with decisions based on varied and cbmplex factors requiring 
understanding of underlying principles and procedures; extensive recall 
of relevant and precise knowledge and experience for decisions and 
Inferences* frequent decisions at the theoretical and abstract level; 
precise and detailed analysis, corf elating , computing , organizing, and 
sequencing of processes or data, as in variable emergency procedures, 
troubleshooting, planning, scheduling, etc; 



FIGURE 6-1 eiassif ication of the Mental Demands Made on Persryrrnel by 
Duties and Tasks 



Source: Air Force Systems Command (1969), 
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Although It is seldoi explicitly stated, the anderi'ylpg rationale of 

mast.o* these classifications is that the job or the Occupation is -a 

given, a fixed quantity. The aim of personnel selection is U.exeflfe to 

find persons who have the abilities, skills, and other characUrr^^e. 

required to perform particular jobs. From the^ «tand>plnt of human 

factors, however, a job is not a fixed quantity but rather something that 

can be modified arid designed to fit people with, Varying characteristics; 

Thus it becomes important to know in what ways people vary auP^y how 

much. In this area there are serious gaps in our knowledge. The most 

thorough translation of indivldaal difference data into deslgS 

requirements has been done i^ the field of anthropometry, which involves 

measurement^ Of the human body. It is possible to write equipment design 

specifications so that the equipment will fit 90 percent. 95 percent, or 

any other proportion of a particular user population. The inf ormation"^ 

necessary to write equally precise d^ign specifications for ot.her human 

' * ' 

dimensions and characteristics, however, is not avaTlable. 

Attempts have been- made to do that, but further research is needed on 

this complex problem. The Air Force's six task dimensions of ambient 

environment, equipment characteristics.^ physical demands .^hazard 

exposure, and task criticality are a good initial effort (see Table 6-1). 

^"^f*'^ Air Force Beslgn^andbddk acknowledges its limitations: "Because 

Of the bro^d range of equipment characteristics, complete criteria are 

not presented here. The following are merely suggested guidelines" 

(.Section DN4C3, p. 13). For example, the manual states that Code 1 

* ^.y^ ^ ■ 

equipment is ^ .. . complex but adequately ^designed foir ^ase of use. . . 

What the definition does not specify is ease of use for whom. 
Sbmethini that is easy for ah astronaut id use may be completely beyOnd ' 
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TAHXf R-1 Clfl^niflcitlon of Equlpfient Ch^iractertitlci und Tfiik Crltlcallty of Virloufi tSukii 



Code Equlpnont Ch/imcterlstlc^ 



.Task Crltlc^Uty 



F.qiilpoK^nt til niffipU litid pre^entl no operltlng or 
mfilntrnflnce problrnii In rcUtlon to the Duty or 
trtftkn; fquipmpnt compicK but ddequat^iy dfiligid--^ 
for fiiiif of iiiie; cqiilpnii^nt slipUfleS tiiilt p«r- 
fbriiincf ; huwiin enjiricering prlnciptei 
^ff^QCtVrty iipptfed Co Htt ispecti; rio fiiturii 
impose n burden on buntfn capablUtlei; etc. 



_* ■ , 

Iluinrth fngincering chamctf rlatlca nnrglnai} 
acrpiiii fcif rfjiiilr or ffplaccnicrit poiiitblf; but 
difficult; noftif controla or dlaplayi violate 
nlnor popoliitton itereotypcs; layout'of con- 
troli rtnd Jifiplays perwlta, btit docs not 
♦ faciiltflte perfonirtnc;?; dispiayi ioderateiy ' 
HlfflcuU to read or Interpret; cbhtfolii Idie- 
viiat dt^tcuit to reach or nanipuiate; etc. 



Kqutpment un§ijitabie_fpr proper taak perfor-. 
fiiincc; IhiiufflclenE liifofwattofi preaented tn 

(iiflpiayi; diaplaya iliegiJi|| cbnUb^ 

firtrendy difficult or Iwpoaaible to reach or 
wfinlpulate; equipment requlrea three^handl to 
operate; acceaa forjiaintenffncfl ext!i?jiely' 
difficult or lnipbialble; etc. 

(Not applicable tb-equlfwicrit chafacfef latlcl* | 



Soiirce; Sir Pbrc^ Systetia Cblniarid (1969); 
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^T/iika that are not critical to the operation of 

pilahed correctly, thcre wlil be no aiittntficant 
effect on the bperatlbhiil capabUltleij of the 
ayatefli or the auqceaa of its designed miaalon; 
^ ifiipropef perforwance iay have soiiie effect on i 
aubaysteS operation^ but_wo«ld not Jeopardlte 
the overall ayatei performance or vlaalon 
aucceaa* 

Tanks that are trltlcai for auhayste" operation 
and may result tn lome iyitei degraditlon if not 
correctly performed; tasks whose failure permlta 
Boae operational capability but degrades the 
applicable aybayatem to the extent that only 
partiaLmissioji auccesa. cHn be achieved; tasks 
, that affect eqtitpment which li tiiportaht to the 
_jppHmum capabillty^^ system but vhere 
Alternate modes may be aelected; task* whoa* 
iltlure would restrict the system In Its 
primary ffllaalon,_b ll^^.P!!^^^''^ ^H* 

selection of other targets of opportunity; 
tasks where matfunctlona night make It 
Impoaalble to dellver*atores by electronic 
actuation but will permit manual delivery* 

taaka that miiat he performed correctly sine* 
they are crlticsl to mission auccess; with task 
ffiiliire the syitem may continue to work (liei^ - 
its bsslc cspabilityi such as flyings may not be 
affected) but lea operattohat effectlveneil ti 
degraded to an unacceptable leyel or iiiaiion 
fulfiiimetit is rttidertd Impbsslbli. 
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Taaija which; If not performed correctly; render 
the ayateii completely Jfibperatlve and Incapable 
pf performing Its mission. 



I9e 



Che: capabilities if an individoai with only an elementary scftooi 
education. To state the problem explicitly, 'i,e do not know exa^Tly how. 
to design complex equipment so that it ca^n be used with ease by people 
with aVferage IQs . people with IQs as' low as 80, pe*bple with fif th-Rradi 
reading abilities, or people for whom English is a second language. 

" • • • : ' 

THE IJ^ACT OF FEDERAL ANTIDISCRIHINATION LEGISLATION 

Antidiscrimination iegiislatidti has focused attention on humae' factors 
. issues related both to complying with legislative reqoltements and- 

maintaining the .productivity of a work force with greater diversity than 
in the past. As a result there is increased concern over the interaction 
of individual differences with programs, such 'as job redesign and training 
as well as over organizational at titudes -toward various populations 
(e.g., the elderly) that, my constrain their performance. ' 

As a result of the U.S. Civil Rights 'Act, federal guidelines have 
been developed concerning personnel decisions that affect protected 
classes, whic^h include: • American Indian or Aiaskian natives- blacks not 
of Hispanic origin, Hlspanics, and Asian or Pacific Islanders. In 
addition, 'federal legislation has made it illegal to ^giscriminate on the 
. basis of sex, age, or disability.. Any personnel action desalting in 
adverse impact against any of these groups can result in litigation. In 
this context, personnel decisions are nS^ limited' to selection or . 
protaotion but rather refer to any persortne'i practice, &uch as job and 
_ workplace redesign, selection for training^, and the use of training as a 
basis for prdmotibn. ' . j /-> I 
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Legal actions resulting from chargers of :«di6crimihati6ri have 
Stimulated research on the procedor(^s necessary to assess the validity of 
these types of personnel practices; however, most of the emphasis has 



to validate -selection tests 



I Ly^^OI^*.*^!. V* ^^^^^^ 

aoglcal, Association, ^^^0). Similar concerns are being 



beert on th^^stablishment of procedures 
(American Psyc^hd 

expressed about methodologies for evalujatiug'^f ainihg and job redesign 
(Bartlett , .1978). The research emphasijs has. been on establishing data 
bases, so that it is possible to desigS programs that do not have adverse 

impact. ■ ■ I 

As a consequence of antidiscrimin^ition le^glslatlon as well as social 
and economic factors, people from special population groups are moving 

into occupations that were previously considered , nontraditionai for 

* _ __ 

them* An example is women who are entering manageria-i and blue-collar - 

jobs and the military services. The military services are also accepting 

njore people (Sale and female) who have lower ability as measured by 

traditional academic aptitude measures. These changes in the composition 

of the work force and the armed services have revealed an important 

problem in addition^ to the human factors issues of deslgnlpg jobs, 

equipment, and training to accqmmbdate individual differences: it has 

only recently been: recognized that organizational attitudes toward people 

entering nontraditionai jobs, may adversely affect productivity by 

^hindering their performance and constraining occupational aspirations. • 
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SEX AND JOF PERFORMANCE 



Sheridan's (1975) description of the American Telephone and Telegraph 
Cbtnpany's experience in placing women in craft jobs illustrates the^ 
implications of human f^^^rs for sex and job performance. Despite 
rigorous recruiting and ccJ^prehensive training efforts,, the women 
recruited into a particular ™job dropped from training at an average rate 

of 50 percent^ and the women who completed training usually did not last 

* 

a full year on the job. A task analysis of^th^job indicated that the 

physical tasks were extremely difficult fSf^wS^en^ to perform; 

f urthermbre', this analysis determined which tasks were causing the most 

difficulty. Some of the most serious problems centered on the use of a 

ladder that weighed approximately 80 lbs. and was 14 feet long before 

being extended. Women had great difficuj^y placing the ladder against a 

building because they had to apply force below the midpoint of the ladder 

just as the force required to raise it' was increasing. A fiberglass tube 

was connected to the top rungs of the iaddjir that enabled the worker to 

push the ladder against the building much more easily i As a result, 

workers who were 5 foot 2 inches weighing 120 pounds were, able to raise a 

72 lb. ladder with one hand. These and other desigr^ modifications not 

^ ^ ^ _ _ . _ _ _ 

only allowed women to perform the job but also resulted in fewer back 

injuries %dr men. ^ 

199 ^ . 




important considerations with regard to age and jtih perfornsanGe are tljat . 
tfie average age of the population is increasing and both age 
dlscrimina'tioh legislation and rulings against forced retirement are 
resulting in a larger number of older people in the work forces Many of 

.these individuals will require additional training as a result of job 

_ ' _ * 

shifts , technological changes, or simply interest in a new career. The 

biases operating against these' people are made obvious by firitton and 

Thomas's (1973) study of the -views of employment interviewers^ They ^ 

noted that 50-year-old workers were viewed as the inost difficult to place 

during a recession, the most difficult for an employer to traii5,'and the 

least able to maintain production schedules. These views are based on 

preconceived beliefs that older workers cannot perform as well on the job 

and cannot easily acquire new skills. Data relevant to thi^se questions 

are virtually nonexistent; a thorough review (Fbzard and Pdpkin^ 1978) of 

perceptual and cognitive data analyzed by age reinforces the view that 

there ar^ few data relevant to work situations. Much of that review is 

based on data from laboratory experiments on topics sych as paired 

associate learnings iconic memory^ and visual discrimination^ making 

generalizations to work situations hazardous at best. 

The deficient state of this research is summarized in Shepp>ard'5 

(1970) generali^^atiohs about basic research on aging and job 

performance: The research fails to differentiate various aspects of the 

work situation^ including physical ^ psychomotor, sensory, and sociai 

^ ^00 ' . " 



r 



10 



characterlstl^cis; most of the emphasis is .oh average jjerformance , with 
little, if any, atte^ntiqn to the substantial namber of individual • 
differences; and ^ there 16 a_ blind faith in trend extrapolations. If 
•w.drkers. ages ;3Q-4P have lo^er morale than workers aces 2(H3b it is 
simply assumed that workers -ages 40r5d wiii have even lower morale. 
A good example of the implications of our lack of knowledge is 

■ - ' ■ _r- 

evidenced by the continuing controversy concerning air\lihe pilot age^ 



health, and-perf ormance . An Inisltute of ^ledlcirie (1981) report notes 
that although the average , risk of acute incapacitation increases with 
ag^, there are large individual differences. In addition ^ vhlle there 

are decreases in capacity, spee^ .o,r accuracy of attehtlbh, memory | and" 

f 

intellectual skills with increasing age, there is also evidence that 

well-practiced skills may not show any age-related decline. The report 

concludes that there is a need for research on age-related changes ambn 

pilots and a need for research on pilot performance oti tasks\that are 

representative of ac-tuai worrk situations. ^ 

- f 
Of more innuediate relevance to this report are the relationships 



between group variables such as age, and equipment design. For example-^ 
as they age, many people require the ^se of bifocals. How does the use 
of bifocals relate to the need to read ihf orma'^tlon from displays such a 
those found on word processing equipment? Is^ it possible that the 
displays must be designed differently or th^t the information must be 
displayed differently depending on the age of the operator? Questions 
such as these fonstitute\a largely unexplored ^opic for research. 
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INTERACTIONS AMONG VARIABLES 



Another' serious gap in bur knowledge is how various individual arid group 
differences interact to affect job performance. For example ^ there are 
considerable data available relating aging to maximum oacygen uptake, 
>7hich determines the capacity of an individual to do prolonged heavy work 
(Astrand and Rodahl ^ 1977) • ^ese data show that there Is a steady 
decrement in aerobic d<5w5T^^^ inning at about age 20, ^uCfi that a 
50-year-61d attains aSigut^70 percent of the maximum of a 25-year-dld - 

Unf ortunately i there are a few data bri mbst pdpulatidri differences or 

* -' -- _____ 

individual differences as they are related tb wbrk situatibris. McFarlarid 

*»- - - - - - ' 

and O'Doherty (1959) concluded the following regarding the relatibhship 
of aging and work perf ormance^yC^lp. 454-455): 

" Although most studies show an unrelieved pictiisx^bf decline in 
capacities, it is well to remember that this constantly changing 
balance between physiological and psychological impairment ^ bh the 
one hand, and increased experience, wisdom,, and judgment, on the 

• other, occasionally results in actual improvement of capacities, 
especially in. those functions which are of greatest importance In 
daily living. * 

i 

These arid other interactions of variables are another almost completely 
uritapped area of research; 



12 



NATIONAL AND ETHNIC CllFFERENCES 



/ 4 



^^^^ ^^^^^^^ 

design that have 4ust recpn^i^w k . t ' 
^ recently begun' to be investigated (Chapanls, 

-73). ,He.e .l«ere.ces a. renectea In anthropometric, physiological, 
psychological, language, and cultural varlaSle. th.t a«ec. e,olp.en. 
design. ( - 

. ^o. «„„ai-s^.. (,,,3) Soto, th^t t.l.sr.pf, ,,,, 

ofelo.u, „ co^.iliiUo. 15 '3ir t«f£lc eootf^ 

howov„. ^^^^^^^ ^^^^^ 

c.™„„lcat.o, ^^^^^ 

E.Sli.h „.= ^^^^ ^^^^ 

.ircr.fe operated 6, E.gU.h-.peafcl^i cuntrloi. Yet there le a 

var.atio. M EosUafi «al.et. a=d pro„„eia«.= , t. the e.test tH.t 

aone dialects, such ii Ehat epbken ii Newcastle' ari „>,f „ 

^ x^ewcascie^ are not anderstbbd by 

people elsewhere in the BrltiR*. tot . 

tn, Brltlah lalesi Obviously thi pfobleas ire ioi-e 

severe »h|a the apeaker's Mtlve laSj^^e la fiot'EoBliah. - 
^=»oll-s«tb.a analysis co«„lcetlo. .rr.r. iidlc.tea that t.la 
l>r..li* can bi aerloua 1. ilr ^traffic c.s.„„lc.tlo„. ..peeUll, „he„ the 
sp..d Of reactlo, la a critical ele.e„t lo avoidlng^n accident. 

»f population dlJfefences Should be ' 

considered In design deelslbne. ' 



Ethnic Variables In Human Factors Engirieering (Chapahls^ 1975) 
provides other exampiles of equipment design complexities, caused by 
language differences^ One chapter (Hanes, 1975) shpws the variety of 
aGCduntlng keyboards that have been designed to accommodate some of the 
European and Mideast languages. Another chapter (Brovm, 1975) ; 
illustrates the design problems that were encountered in designing a 
cdniputer terminal for Japanescs, a language that is markedly different 
from the Iridb-Eurbpeah languages. Xn general^ there is little r 
appreciation *bf the prbblems involved in designing equipient for diverse 
national and ethh^ grHiups. The Human Engineering Guide to Equipment 
Design (Van Cbtt and Kinkade, 1972) is the best single scJurce of human 
factbrs data available, yet ft is almost entirely concerned with Aierican 
and Eurbpean data. It is necessary to learn to what extent its data and 
design recommendations heed to be fibdified or supplemented for } 
interna tlohal use. 



; • INDIVIDUAL DIFFERENCES AND TKAIXING 

Closely related to problems of equipment design are thbs^ assbciated with 
\^the training of individuals to bperate cbmplex "equipment . Here again bur 
information is seriously deficient. An apprbach that has sbm^ prbmise Is 
the aptitude-treatment interactibn (ATI) tribdel. The gbal bf this 
approach is to match a particular mbde bf Instructibn to an individual's 
distinctive characteristicis so that each persbri i^ assigned the most 
appropriate learning procedure. A diisdrdinal aptitud^-treatmehf 
interaction is one in which indivi^duals with high aptitude perform best 
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with one treatment (e,g,| training or display), while those with lower 
aptitude perform best with another treatment. Thus, the aptitude level 
of the individual determines the form of treatmierit that has the best 
crfance of success. Aptitude in this context refers to any personal 
characteristics that relate to learning and so can Include a broad range 
of variables, such as styles of thought, personality^ and various 

slkdlastlc^ aptitudes . . Treatment has typically referred to Instructlbnal 

_ _ 

modes like programmed instruction, computer-assisted Instruction^ visual 

versus verbal presentations, NCtc; it can be generalized^ however, to any 

intervention, including job redesign. 

An exhaustive review of this appealing strategy is provided in the 



text by*Crdnbach and Snow (1977). They exaiined a lar^ number of 
potential aptitudes, such as teafalng rates, abilities ^ ^nd personality; 
arid considered their interact^^_ with various Instructlbnal technique's. 

While early reviews of this to^ic were more pessimistic i Crbribach arid 

_ _ ' _ 

Sribw's extensive review and reanalyses of data have led them tb cdriclucje 
that aptlt^ide treatment interaction effects are real pheribmeria • They 
ribte that the findings that most clearly suggest ATI effects are those 
dependerit bri prior learning experience: ^e technique that wbrks best is 
the' one that ari Iridlvldual has already experienced- However^ ATI effects 
have not ofteri beeri generalized or replicated. Goldstein (1980) ribtes 
the need fbr systematic empirical and theoretical research that matches 
individual dlffererices among learners to^* various instructional 
strategies. The haphazard asslgrimerit bf Individuals with particular 
abiitties to any available iriStrUctibrial technique is not likely to 
produce dividends. 
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BARRIERS TO SUCCESSFUL PERFORMANCE 



Another important topic the Identification of harriers to successful 
performance for different groups. For example, some employment^ 
interviewers perceive woyien as more likely td be absent and to have fewer 
skills, even though they have no evidence to support these beliefs 
, (Brit ton and Thomas, 1973). Similarly, the elderly are viewed as 
difficult to train (Brit ton and Thomas, 1973); Researchers concerned 
with these Issues emphasize that the identification of organizational 
/constraints, in military organizations for example, is a first step in 
understanding and riesblving their serious -retention problem. One study 
(Boyd et al., 1975) of 1,573 women in their first tour in the Army's 
basic training program was critical of the program 's failure to provide 
realistic expectations about the training process. Subsequent to the 
basiis training prbgr^^ supervisors reported the main difference between 
good and poor performers was Job-related attitudes (discipline^ following 
orders^ military courtesy) that were hot adequately presented in basic 
training. . ' * ^ 



RECOMMENDATIONS FOR RESEARCH ON POPULATION GROUP DIFFERENCES 

A research program to explore issues cbhcerhihg population group and 
individual differences would heed to take several approaches: 

(1) It is hecessary to cbhduct literature reviews and examlhatioti; 
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of feportB th.t forecast which type of populattor, group variables 
_(soch a. .ge and sex) and which type of wo?k situation parameters 
(such a. Visual displays o. a Word processor) wiii be important i. 
the future. ^ - 

(2) It is necessary to collect and examine .vail.ble theories and 
empir/e.l data. about the relevant parameters (e.g.. changes in 
information processing capability as a function of age). 

(3) Research should be sponsored on a number of topics: 

o The relationship between popuiatioti group variables and 

performance' on relevant work tasks. . - " 
o ^ The inter^tion between popuiatioa group differences and 
various interventions, such as job redesign and' training . 
b The specification of design chatiges based on research 
-findings resulting from these research recommendations. 
(4) In addition, data should be collected and analy2ed to identify 
Mnd remove organizatlotz^l constraints that serve as barriers to the 
successful performance of various population groups.' such as wome. 
and aged and handicapped people. 
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VII 



APPfclS n >gTHOD S IN HUMAN FACTORS 

4 



As part of an engineering team, human factors specialists apply their 

koowledge and skiiis to system definition, desj^n, development, and 

eva^^ation in order^ to optimiie the capabilities an^ performaoce bf 

human-machine combinations. Their task can be foriidabie in cotsplex 

^ system development. For example, military standard HIL-H-46855B of the 

< _ V. 

Department of Defens'e details the human factors requirements that must 

addressed in the development of military systems; an outline of these 
requirements appears as Figure 7-1. The outline ±s also a reasonable 
representation of the human factors considerations that may be relevant 
to the development of any system. ^ 

In designing arid creating systems human factors specialists Use a 
varle^ of arialytic and data^gatherlng techniques to assess problems, 
develop machine arid human requirements and functions , arid evaluate syste 



The principal authors of this chapter are Alphdrise Chapariis and Robert T 
Hennessy. It is based on a workshop on applied methods held in December 
1981 under the sponsorship of the Committee on Huinari Factors^ The 
workshop participants and, therefore, the principal contributdrs to this 



Parks, Boeing Aerospace Company; Richard W. Pew, Bolt Beranek & Newman 
Inc.; Erich Prien, Memphis. State University; Johri B. Shafer, IBM 
Corporation; and Robert T. Hennessy, National Research Council! 
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3.1 General Requirements 

3.1.1 Scope, and Nature bf Work 
; o Analysis 

O Design/Development 
^9 Test and Evaluation 

:A 3.1.2 



3.1.2. 
3.1.2.2 



1. 
2. 



Human Engln|ering Pfogram Plan and Other Data 
1 B^man Engineering Program Plan 

3:1:2:1 OtKfit:-''^ '""^^^ Engineering Progr.„ Plan 

3.2 DeJiL .Sui^i;;tr"° ■ 

3.2.1 Analysis 

3IIII Processing Analysis 

3.2.1.1.^ estimates of Potential Operataf/SlataJner 
^ ^ 1 i , Processing Capabilities . 

T V Al^°"tion of. Functions 

19,; \^Egulptaent identification 

■i.Z.l.J ^^eal^sls of. Tasks 

^'t'^'UI . Analysis of Tasks\ ^ 

^m^^m^^d " 

Sp.°Mlftr'^ Ri,.lr.i.iis BO mt E^c.ed a„i„ 

3. Jnput Data far ^ 

o Preliminary Maaning Levels 
o Equipment Procedures • n 

o Sklll/tralnlng Requirements 
o- Communication Requirements 
^. Critical Human Performance 

5. Possible Unsafe Practice 

6. Promising I.provet.er.ts in Operating Efficiency 
i: Ide'ntlfy?^!'^*^^ ^'^^^^"^ ^-^^ 

• ° Cue's""'""" ''^'"''^^'^ ^-'^^"^i^g Task Initiation 

o Information Available to Man 
b Evaluation Process 

b Decision Reached After Evaluation 
o Action Taken 

b Body Movements Required by Action 

Wn^^f-" Envelope Required by Action 
Workspace Available 



b 
d 

o Locatlon/condltlbn'of Woric Epvlrbnment 

o Zrequency/Tolerances for Action 

o Time Base 

b Feedback Action Adequacy 
Tools and Eqalpment Required 
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t> Number of Personnel Required arid Speciaities/Experlerice 
6 Job Aids/References Required 
6 Special Hazards Involved 
ro Operation Interaction Where More Than One Crewman Is 
Involved 

6 •Operational Limits of Man (Performance) 
6 Operational Limits of Machine (State-of— the-Art) 
2. Covering All Affected Mission/Phases ^ Including Degraded 
_ _ > Modes of Operation 

3.20^.3.3' Loading Analysis . , - 

1. Individual Crew Member Workload Analysis Compared with 
Performance Criteria 
_ 2.' Crew WorWoad Analysis Compared with Perfbrmarice Criteria 
_ 3.2.1.4 Preliminary System and'Subsystem Design . 
3.2.2 _ Human Engineering Studies , Experiments and Labdrat^y Tests - 

3.2.2.1 Studies, Experiments and Laboratory Tests 
3.2.2.1.1 Sockups and Models 

_ 3.2.2.1 .2 Dynamic. Simulation 

3.2.2.2 Equipment D^ail Design Drawings 

3»2-.2.3 Work Environment, Crew Stations and Facilities Design 
o Atmospheric Conditions 
o Weather and Climate 
o R^nge of Acceierative Forces 
o Acoustic Noise, Vibration add Impact Forces 
o Provision for Human --Performance During Weightlessness 
? ^?'9Y^^^®? Minimizing Disdrientatloh 

o Space for Crew, Activity and Equipment ' 
o Physical, Visual and Auditpry Links for All Man-Equipment 

Interfaces * 
o Safe, Efficient Walkways , Stairways ^ Platfprm^^ Inclines 
o Provision to Minimize Psychdphyslblbgical Stresses 
o ProviBlon*"td Minimize Fatigue--Physical , Emotional^ 

o Protection from Hazards— Chiemlcal , Bidrbgical^ 

Toxicological, Radidldgical, Elpctrical^ Electromagnetic 
o bptimani liiuminatldn Per .Visual Tasks 
o Sustenance ; Storage and Sanitatidn 

o Crew Safety Protection Relative to Mission Phase and Control* 
Display Tasks 

3.2i2i4 Human Engineering in Performance and Design 
Specifications 
3i2;3 Equipment Procedure Development 
^•^•^ Human Engineering Test and Evaluation 

3;2;4.1 Planning 

3.2;4;2 ^ Implementation (Include As Applicable) 

o Simulation or Actual Conduct of Mlsslon/Wofk Cycle 
p Human Partlclpatloh Critical to Speed, Accuracy, 
Reliability, Cost 



FIGURE 7-1 Continued 
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° Sf"""'*'>''^J«°Pi« °^-Non Critical Scheduled/UnBchedulfed 
Maintenance Tasks- ' 

0 Proposed Job Aids 

1 ^^P"«^°tat±ve user Personnel. elSthing and Equipment 
o, Task Performance Data Collection en Hmenc 

^ b Tasfci Performance Discrepancies—Required vs. 'Obtained 
o Criteria f of Acceptable Performance ' 
3.2.4.3 ■ Failure Analysis (Human Error Factors) 
VI 5.2.5^ Cognizance and coordination | InterdisSpiinary Integration? 
3.3 Data Requitements Per £ontract Uata List gracion; 
3.5 Data Availability to Procuring Activity ; 
Drawing Approval by.MFE fo^ Man^Machine interface 
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FIGURE 7-1 eonciaded ^-~sT ^ 

Soutffce: Adapted from Parkis arid^^pririger (1976). ' 
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or subsystem performance. Although many of these problems wbiild ideally 



be solved with the experimental isethods used in scientific research^ 

___ J' '_ ' 

practicing human factors specialists rarely have the luxury of usin^ 

properly counterbalanced experimental designs ^ with a range of levels of 

factors and the precise control of unmariipulated variables.^ This is npt 

. T 

to minimize the importance of experimental methods which are' used 

whenever possible and have provided much of the basic data in human 

factors handbooks. However ^ applied methods are necessary both as 

Buplements to experimental methods^ e-:g.i for problem analysis and 

structuring, and as substitutes when the pressures and constraints of the 

engineering design envirdnineht preclude experimental investigations. 

Most practical work in huiban factors is done under conditions tha^ 

involve ^he incomplete specification of system functions^ complex 

combinations of conditions that cannot b*e separated or controlled, 

^stricted sets of alterpatives ^ limited time apd opportunities for 

investigation, and pressure to produce definitive results quickly. From 

necessity, human factors specialists have evolved an armaSentarium of 

applied methods that are appropriate to these conditions and that are 

unfamiliar to mOist academic researchers. T^ese applied methods are 

formal means for acquiring or organizing information about human factors 

\ ■■■ - 

characteristics that arise in the context 3©f system design, development, 
and evaluation. » * ^ 

Applied methods^re diverse , reflecting the many purposes for which 
human factors information i^ used. Some bf them come from psychology^ 
for exanjpie, questionnaires and techniques for acquiring, summarizing, 
and'' analyzing data. Some^4=i^e been borrowed, with o^ without 

t 

_ iliodificatton, froin other fields, such as industrial engineering and time 
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and motion engineering. For example ^ analytic me^bds draw heavily on 
the engineiering practice of systems analysis ^ which identifies inputs - 
outputs^ the functions performed^ the range of values that variables may 
assume, fjrdciess flbw^ the sequence of events^ and the timing of the 
interrelations of system cbmpbnehts. Other methods, such as th^ critical 
incident technique; and link analysis,, appear to have been created by 
human factors specialists to meet their needs in solving p>articular 
problems. 

Whatever their origins^ applied methods have been developed as 
tbbls to help answer questibns when there are constraints of time ^ 
dollars^ and freedom of action and when experimental methods are hot 
suitable to answer the questions that arise in system development. 
Although it is characteristic of applied methQds that they make it 
possible to acquire ^nd prepuce data and information, only to the degree 
of resolution and reiiability sufficient for a particular purpose, these 
methods are systematic- and 6b jective^procedures • That is^ the procedures 
are repeatabie and input and" output data are operationally defined. 

The importance of applied methods in human factors work is clear 

from the number of technical reports and journal articles that discuss 

^ _ ________ _ _ _ • 

one or more appliied methods. Two recent reports (Wliiiges and Topmiller, 

1980; Gieer, 1981.) list hutoan factors procedures necessary for Air Force 
system analysis, design, and evaluation; the latter report gives brief 
^^^^criptions and critiques of approximately AS^huma^ engineering 

procedures, the majority of which are applied methods. Figure 7—2 lists 
applied methods tfiat appeared in keyvord lists of articles published ; 
between 1976 and 19^1 in Human Factdri^ the journal of the Human Factors 
Society i 217 



Accident studies ; 
* Attitude studies 

Critical Incident studies 

Delphi techniques 

Fault ti%e analysis 

Functional analysis 
;Lat>se tlxne photography 

Near-accident studies 

Operational sequence analysis 
^/^qulrements analysis 



Activity analyses 
Cost-benefit analyses 
Decision analysis 
Failure mode analysis 
Flbv analysis 
Job analysis 
Link analysis ' 
Network flow analysis 
Ques tionnalre^^ 
Task: analysis 



FIGURE 7-2 Applied Method Names Appearing in Keyword Llst^ of Articles 
Irp ^uman Factors Between 1976-1981 
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Despite thlis side variety of applied methods, there Is general 
agreement among human factors specialists that we need to Improve 
fisting methodjs and develop new ones (Topmiller, 1981; Melster^ 1982). 
Advances in technology, particularly in th^ speed, power, and memory of 
cotnputers, have generated concfern recently with the human factors 
^ .eleiaents of computer software. At the same time, the explosive growth of 
, computer use, with resultant increases in the complexity and integration 
of System components, the autdSation of fonctidns, and the use of 
artificial intellig^ce, all hai^prof ound methodological implications 
for the analysis And description of the role of humans and computers in 
such systems . ' : 

Applied methods have never previously been treated as a single 
topic deserving attention in its own right.* Cdnsequentlyi information 
has hevpr been gathered on the number and varieties of applied methods 
available and the frequency and adequacy with which they are used. The 
workshop held by the ebmmittee on Human Factors, on which the discussion 
in this chapter is based, was an attempt by committee members add a group 
of acknowledged experts in applied methods to identify problems and needs 
with respect to applied methods. Even in the absence of data on the 
variety and frequency of use of applied methods, we have been able to 
identify several major problems and to recommend solutions, which may 
make substantial improvements in practice possible. Three major problems 
are discussed: (1) the lack of adequate documentation; (2) the limited 



*This situation contrasts with experimental niethbds^ for which there are 
many textbooks and source books for readers at all levels of 
sophistication. 



opportunities available to learn applied tnethods, either in colleges and 



unive^ities offering hutnan factors courses or as part of the cbhtiriuirig 



education of human factors specialists; arid (35 the lack of research to 
improve existing methods and to develop new methods that will provide the 



data and information needed in current and future practical human factors 
work; 



accumulatibri of kribvledge* Humari factors projects (i.e.^ participation 
in the design of systems) arid' the sblutibri bf special problems come arid 
gb in great variety. Topically wbrk is p^rfbnhed^ reported ^ arid 
forgotten as new systems, arid prbblems develop. Codified^ archival 
repbsitbries bf practical wbrk--i.e,i review books arid articles that 
summarize the kriowledge arid procedures used iri human factors applications 
to some po^nt iri time' — are rate. As a result the historical memory of 
human factbrs methbds resides largely iri the heads arid in the report 
files bf practitioners. By contrast, in the literature on scientific 
research^ the methbds used by irivestigators are maintained and 
dissemiriated in the curricula of university departments and preserved on 
library bookshelves. 

As an important first step. toward imprbvirig kribwledgfe about arid use 
bf applied methbds^ we therefbre recbmmerid that one br more projects be 
initiated to cbinpile arid review the available infbrmatibn bri applied 




DOCUMENTATION OF APPLIED METHODS 



The practical wbrk bf humari factbrs specialists^ urilike scientific 



research^ does nbt result iri an orderly progression and an orderly 
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methodologies used in hansao factor, and related fields, such as 
Industrial and organizational psychology, personnel selection, and 
instructional psychology. The object of the review would be to determine 
what method, .have b«n used, hpw they have been used, where they are 
used, and what th.ir advantaged and disadvantage, are. The project 
should .ISO include a critical arSlysls of the methods. Other purposes 
of the review would be to structure or codify the methods and to document 
them for subsequent educational and research purposes. 

It would also be extremely valuable to pr/actitidners , eduSatori, 
and researchers in hunan factors to have a compendium that codifies and 
provides standard or generic descriptions of applied methods that are 
used in practical human factors work. Development of such a compendium 
would require a great deal of judicious and careful effort. One of the 
primary difficulties would be to decide which methods are viable, valid, 
and Useful. Because such a compendium would necessarily be an lipiicit 
endorsement of the methods described, we recommend that eight criteria be 
used in the S^ection process. Methods that meet the criteria listed 
below could be regarded as having sufficient stature to be of value in a 
variety of human factors applications:, . ' 

• ; 

I 

Importanc^>--^Does the method produce needed Information? 
Cbst--is th/ method efficient in terms of effort and timet ; 
Utility— Can procedures for Using the method be easily interpreted 
and implemented? 

Available input Parameters— CfiH the necessary data be collected in 
a direct, objective, and reliable way? 
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Usable Output — Does the method produce results that are 

Interpretable and useful for decision making? 

Validity/Verification — Can or has the method been found to produce 
the information it is supposed to? 

Theoretical Foundation — Is the method Supported by accepted 
behavioral or measurement principles? 

Robustness — Can the method be applied to a variety of problems or 

V- _ 

in different contexts? 



These criteria imply that the approach to documenting standard 
definitions of applied methods should be conservative. That is, only 
those methods for which th^sire is evidence of practicality an<l/ validity 
should be selected for inclusion in a compendium. Methods used in 
workload assessment provide an example of the Importance of using these 
criteria. Meas,urement of workload is a current topic of intense research 
interest; cbnseqiiently a. large number of theories, approaches, and 
positions have been put forward. Since most of the recent work has not 
been validated through practical application^ it would be inappropriate 

to describe them as slbahdardi accepted methods. Older methods exist for 

' - _ - - / 

assessing imposed workload that^ while perhaps wanting in certain 

_ . _ _ _ 

respects, have been proven through repeated use to be practical^ 

relj^ble^ and valid (Parks and Springer^ 19765 and are likely to meet bur 

criteria. Nevertheless, there will be hard choices to make in deciding 

what constitutes an accepted, standard form of a method. 

Multiple variations of a method should probably not be included. A 

compendium that includes only a set of core methods that meet the 

criteria would be of great value for both practical work on system 
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."U-n.. .i con.,., .„a ,,,.„^„„^ e..p,e.e„a,. ..v..,,." 

.o..e,ae„U, .„MMtio, H. .«.,„„e.. * ,,,,, 

e.eh p..„.^, ^^^^ ^^^^^ ^^^^^^ 

Eeca„„ ,e,„l,e„e„„ c.n.Si.Ut. « . pa.ticai.r- p„j.A. IS 

.he addition.! dOcU«„t.Uo„ .„d .i „ 

: th. standard oethodi can be carried iati 

In the coufie of compiling . Reasonably cofpreheiiiie Hat of the 
»ast sen«ill, tno™ appu., ^^^^^^ 

that the Bithodoloiie. could bi gr.npid iito fiv. c«esbii« according 
th^r purpose. MVe categories of applied methodoloileS iiii 
•H^opriate: iSilysis, identificitlin of need., diti «Uec£io., 
predlctio«. and evaluation. Each iethodology app.iri oHi under On* 
H...i«g. iab<,.,j, ,j appropriit^ to iof. than on* 

category. 

The organization of Figure 7-3 is probably a useful guide to the 
.cope of work Involved In documenting applied methods. >e citigorles ' 
reflect a sequent of methods .used. fr.. the early concept definition of 
-a syste. to It. .v.lu.tlon. there Is al^ a rough correlation b.t^en . 
the difficulty -and d.tall Involved In particular ^e^hods .nd th. stage of 
application In the process of system development. 

Documentation Of applied met}.odS nicessa^ly require, r.vlew of the 
technical ilteratur. to. extract descriptions' of' applied method.. To 
^^pect a iingl. or . .n,all group of .xp.rts to, adequately r.vlew and 
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ANAtYSiS 

System Analysis 
Function/ Task Analysis 
Iiifdrxziatidn Analysis 
Scenario Analysis 
WorKload Analysis 
Tixne-Line Analysis 
Operational Sequence Analysis 
Failure'^bde Analysis 
Fault Tree Analysis 
Link Analysis 
Function Allbcatibn 
Anthropbibetric Analysis 
Decisibn Analysis . 
Display Evaluation Index 



IDENTIFICATION OF NEEDS 

Critical Incident Technique 
Surveys/Quest ibnriaires 
Accident Iriyestig^tipn 
Interyiews/Grbtip Techniques 
Defihitibn bf User Pbpulation 



DATA COLLECTION 

Activity Analysis 
Time Lapse Photography 
Real Time Film/Video Recording 
Direct. Observation ^ 
Physiological Recording 

Quantitative Performance Recording and Analy 



PREDICTION 

The Human Error Rate Procedure (THERP) 
Data Store 

Human Operator Simulator (fiOS) 
Cbhtrbl ^Theory 
Accuracy Theory 
Predetennined Time Analysis 
Readability* indices 



FIGURE 7-3 Generally Known Applied Methods Categorized by Purpbse 
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EVALUATION 

/T^^ic Piaa Evaluatlbh 
' Simaiatloii 

Mock- Up 8 

Walk Thrbughs 

eheek. Lists V. . 

Ratings^ ' 



FIGURE 7-3 eoncluded 
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dbcmbetlt the entire range of applied methods would be impracticai; a more 
feasible approach would be to subdivide the work according to the five 
categories of purpose. The Individual tasks would thereby be indrc 
ti^actable and make better use of the skllls^bf Individuals whose 
Tcnowledg6 and expertise Is likely to be confined to a single category 
rather than ^he full range of methods. This apprb^ih would also allow 
the work on each subset of methods to be performed cbhcurrehtly. ■ 
Whatever the approach taken, prbduclhg a cbmpehdlum of stahdard^^'iisable 
descr^tibns of proven applied methods wbuld be ah extremely valuable 
contribution to the field bf human factbrs and cbhseqiiehtly tp thej future 
development of humah^achlhe systems. 



SURVEY OF HUMAN FACTORS SPECIALISTS ON APPLIED METHODS 

Because of the dearth of. Ihfoinnatibn on the variety and use bf applied 
methods in human factors work we recommend a survey bf human factors 
practitipnerB concerned with the acquisition, design^ develbpmeht ^ and 
eva-iuation or modification bf equipment and systems. Such a survey would 
detertnlne the importance and frequency bf use bf existing applied methbds 
in their work; the kind of information most needed in human factbrs 
applications for which existing applied methbdblbgicss are Inadequate br 
hbhexistent; and the methods for which descriptions and guidance for use 
are most heeded. 

The survey wbuld prbvid^ the necessary ihfdrmatidh on which to. base 
dbcumentatiohi educatibh^ ahd research efforts. Review, codification, 
stahdardizatiohi and dbcumehtatlbh bf existing' methods shquld procesi"!; 
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according to the priorities of importance and frequency of use derived, 
from the survey"?^ information from the survey would be useful in shaping 

— - - 

human factors curricula in colleges and universities bo that students can 
be trained in applied methods that they will subsequently need on the 
Job. The continuing education needs of human factors specialists could, 
also be meS by means of tutori^s and symposia On the applied methods for 



which there i^ the greatest need for informations Finally, the results 

p - : - : - -- : 

of tyt€ survey would provide a sound basis for basic research efforts to 

extend or improve existing methods or develop new methods (to meet these 

_ • ^ 

needs ; 

f 

Construction of the survey instrument itself .would require a review 
of the technical literature for descriptions and definitions of applied 
methods^ which the survey recipients would be expected to rate* The 
i'iterature review would also provide additional data, complementary to 
the anticipated survey^ on the variety and frequency of use of applied 
methods reflected in the technical literature. A product of this review 
would be a relatively comprehensive bibliography of technical reports and 
Journal articles that discuss applied methods in more than a cursory 
fashion; this bibliographic information would be extremely valuable for 

subsequent efforts oh the codification and dbcumehtatibn of existing 

I 

methods and the initiation of research efforts to extend these methods or 
develop hew ones. 
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EDUCAtiON IN APPLIED ^^THODS 



Education In Colleges and Universities 



The absence of codified information and the lack of easy access to source 
reports inhibits instruction in applied methods at colleges and 
universities that offer degree programs or courses in the field of human 
factors. General hunian factors textbooks give at best o^ly a cursory 
overview of a few applied iethods and present case study examples that 
highlight the subBtantive issues and results rather than the methods* 
There are no texts suitable either for cbllege^level instruction or as. a 
reference for practicing human factors specialists that adequately treat 
applied methods. The single exception. Research techn±que s_^n_ Human 
Engineering (Chapanis, 1959), discusses only a limited set of methods^ 
For the most parti instructors mUst j:eiy^ on their own experience and the 
descriptions of applied methods gleaned from the technical literature to 
develop coursje material. They have no current and cdtnprehensive 
reference works to develop a balanced and thorough dburse in applied 

methods. « 

__ .__ _»__ 
Human factors worlt is diverse and is performed in many 

settings — ^^i^ei, military research and development centers, other 

government facilities, and commercial organizations. Ideally^ 

instruction in applied methods would emphasize the methods of most use in 

real-life settings. Without data on the variety and frequency of use it 
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Is difficult to decide which applied fflexhods shbuldlbe taught in human 
factors courses at the undergraduate and graduate J^els. Clearly the 
development of a compendium of applied methods, £s recoiiended in the 
previous section, would be of substantial benefit for formal eduqafci5nai 
purposes. Until such a compendium exists and survey data is compiled on 
the variety, frequency of use, and capabilities of applied meth5d6, no 
meaningful recommendations can be made to improve education in applied 
methods i^n colleges and universities.^ 



Continuing /^d^catidn in Applied Methods 



equal concern is ^the lack of suitable contisuing education courses in 
appiiedv methods for practicing human factors specialists-, 
the problem of inadequate methbdolbglcal preparation in formal education 
extends to the work setting. At present it appears that many presumably 
well-trained htiiiiah factors specialists work witKout adequate knowledge of 
applied methods, and what knowledge they do have about these methods is 
acquired on the job. - 

. . Currently employed human factors specialists could benefit greatly 
from continuing education in applied methods specifically related to ' 
their current work. Development at colleges anj aniversitiBs of 
educational programs in applied methods that provide a thorough treatment 
of a range of applied methocis would require a substantial amount of.; 

planning arid course design work. Undoubtedly the broad inception of 

- - - I 

these programs, and the realization of thei^ eventual benefits in 

practice, will be some time in cdmingi ' Unlike formal education in 
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applied methods, however^ the development of courses for cotitinuing 
education cooid be done more easily and produce xnore innnediate positive, 
effects. Human factors professionals are likely to be more easily 
educated because of their general knowledge of human factors techniques 
and the likelihood that they have at least a working familiarity with 
some applied methods. Because of their pre^loi^ education and 
experience » continuing education courses fo^ them can be mach more 

practical, with less emphasis on theoretical foundatibDS. - Based on the 

/ \ 

membership of the Human Factors Society, whith numbers neafly 3,000, a 

_ _ ' _ . _ r:. _ _ _ . _ _ 

reasonable estimate of the actual number of practicing human factors 
specialists in this country. who could benefit from continuing education 
in applied methods is between 5,000 and 10,000. 

Fostering and promoting continuing education by means of tutorials 
on applied methods is one of the most important and immediate ways to 
improve the field of human factors. Moreover, this kind of activity 
could most easily be initiated by military and other federal agencies 
charged with advancing scientific and engineering knowledge and 
practice. These tutorials could directly benefit human factory 
specialists employed by the gdveriioient as well as those employed by 
civilian' organizations that develop equipment and systems for the 
government. It is therefore recommended that ihifial tutorials on 
applied CDethods be developed and conducted under the sponsorship of one 
or more government agencies. While we suggest methods to be discussed in 
the tutorial below^ it would be more prudent to base the choice on a 
needs analysis of the data derived from the survey recommended above. 
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Such a tut.tl.l could se^ve ....r.l purposes beside, the obvious 
Pne of improving the professional competence of hu«.n factors 
specialists. First, the .aterl.l. generated for the tutorial would 
contribute to th. development of standard definitions .nd documentatio. 
Of applied methods, since the eour.. materials would H.v. to describe the 
subject method. .1th sufficient ..r. and detail to alio, human factors 
specialists to u.. them easily and pro^rly. Second.' the tutorials would 
be a means for v.lld^lng a prior ...<i^nalysls of which appUed ie^od. 
are considered most important to hu^.n factors practltl.n.rs, Attendant 
at the tatorl^l. .ould also help .a..er a more fundamental question: Is 
there genuine Interest in learning about applied methods? Third, the 
initial tutOrl.i would serve a. a tist to evaluate instructional methods' 
-&nd course structures for training m .the uie of applied methods. 

\t is suggested that the Inltlai tutorial should consist of thi^ee 
parts: (1) in introductory review of the applied methodologies within 
each Of the five catego^s listed in Figure 7-3, (2^, a comparison of 
techniques within each ca\^gory and a discussion of how to select the 
appropriate method for a particular application/ and (35 detailed 
instruction and practical work on a few selected methods. We suggest 
five particular methodologies as subjects for the. initial tutorial: 
Task anaiysis; 

Time line analysisi ^ " . - 

Activities analysis; 
Simulation; and 
Information Analysis, 

23i 
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Because these methods as well, as others are either poorly or 

^ . ■ _■_ __ __ 
Inconsistently defined » brief definitions of the five methods recbinibended 

» ■ ■ 

for the first tutorial are given In Appendix A. It would not be 
practical to cover more than five methodolbgles at the Initial tutorial; 
five may even be too many. 

There are a number of other ^pecif^lc cbncerris relevant, to the form 
and development of a tutorial oh applied methods. Experience has shown 



tutoria^ to be otily the first step in learning to use a particular 
technique properly, Geherially^ ah Individtial needs several days of 
supervised application ; to become competent in usihg^ a particular method. 
Therifore^ the tutqrlal should not be simply a symposium but rather 
should be a workshop in which the attendees could gain hands-on 
experience, A by-product of the initial tutorial would be the " 
development and testing of the structure ahd effectiveness of the Ihltlal 
instructional methods. 

A tutorial on applied methods would probably require 10 to 40 bours 
of plahhihg and preparlhg for each hour of Ihstructlbhal time, sihde the 
tutorial should ihclude practical workshop exercises in addition to 
lecture^ a good part bjthe effort of preparatioh would have to be 
devoted to development of materials i It is likely that the practicum 
would require dtie or mote assistants in addition to the instructor. 

An individual or small group should be selected to develop a master 
plan for the tutorial workshop. The primary goal would be to choose the 
methods to be taught in the tutorial. This detertcilhatibh should be based 
largely on the' heeds ahalysls of the data gathered from the methods 
Survey of humah factors practitioners recommended abr'«»e. The Ihdividual 
or group should also address such issues as the humber of days the ^ 

-'■'^■■.'r ' 232 
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tutorial should rubi whether it i □ c 

. wnether it should be conducted independently or tn 

^association vlth a national meeting, the estln,ar.H \ 

estimated costs, and the 

selectiSn of instructors.' 

. rt,.„o» .Bvl... ^^^^^.^^ 

»u...... ^^^^^^ Be «k 

POio.. .n,,„,,. ,„ ,.a.e„.. „. 

- o.. =PPU.. ...H.a. i„ ,,,,,, 

t.., ,„ ^^^^ 

.bl. i.n„..ce p„..,«, ^^^^^^^ 
t..e..„ . ^^^^^ ^^^^ .^^^^ 

»n .o ^^^^^^^^ 

ana h^„ , p,,,i,„,,,,^ ^^^^^^^^ ^^^^^^^^^ 

b.e*u.. _ ^^^^^^^^^^ ^^^^ 

...,ua.e ea„c,U0„ .ppUea .»H»a. i„ .oUe.e. „a uMver.Hte.. 

TH. t«orl.I .ppf.p„.,, Hui.„ p.<^f^«.„,*i. 

ba s.u,M. .^S., ^^^^^ 

be ..nefloial .o Hu.ip fi.,.., sp«i.n.... H 

a».Igo chers taUorea t<, .he Mckgrouna. .pa „«a. of fi... oth«. 
jrovps. «. reco„e.a thi. .u^.H.I. f.. .h..e .Efie. groups aeVelopea' 
flr« for e„g,„«rs a„a »ub.e,„„tly fp, r.i.l„i„s ,,<,ap.. 

rcr .1, ,„aie„«. th. .uipf,.!. .„.„:a be. repe.^a .t ..v.r.t Vi.., 
ana l.c.tio„. 6„ih to the experience iv.ll.&l. „ .11 ^lip 
interested .„a to recover the initl.iiiveldpiiSt cost.. ' '' : 
: U IM - 
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RESEARCH ON APPLIED METHODS 
* * 

Each applied method was originated to fill sbine particular need for 
infbnsatibn to support system design^ evaluation^ or problem analysis. 
Through a succession of repeated » successful use in different contexts, 
methods have evolved and have become known and accepted as tools of the 
trade in human factors work. Because they were developed as a means to 
some practical end and so vary In fom^ depending oh the situations in 
which they are used, there has never been very much concern about their 
refinement or extension. That is, an applied method has rarely been 
regarded ''as an important topic worthy of research investigation in its. 
own rights independent of a particular use. This lack of status is 
partly refl€5Cteii and partly caused by the absence of standard 
documentation of applied methods* In additibh^ the people who use 
applied methods are practitioners and, in some sense, generaiists in 
human factors rather than specialists itt methodology. There is no body 
of experts who deybte their careers to the study and development of 
applied methods rather than their actual use, as there is for 
experimental design and statistical analysis^ 

Applied methodjf, however, are the principal means by which human 
factbrs work is accomplished.. In light bf ^^I^^ cbhtibutlbn to systems 
wbrk^ applied methbds are a sufficehtly impbrtaht topic tb deserve 
research attehtibn. Advances shbuld hbt depend solely bh incidental 
effbrts made by human factbrs specialists in the cbUi.*>e bf their* wbrk. 
Basic research specifically devoted tb the validatibn^ refinement ^ and 
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extension of existing methods and to the develbpmeht of new methods ±i 
essential; 

Improvement and Extension of Existing Applied Methods 

• . ■__ ■ . 

As previously discussed, fundamental probieass are the lack of documented 

definitions and descriptions of existing applied methods and the lack of 

knowledge about what information is needed . in human factors work. 

Dbcumentatibn and survey work is necessary to provide baseline 

■ ■ f 
descriptions and to help identify the particular problems and 

shortcomings of existing methods. 

Without this information it Is difficult to specify what research 

on which par?tfeular methods would have the greatest value in terms of its 

contribution to the improvement of human factors work. Nonetheless^ we 

propose some ousting methods «s subjects deserving research attention 

because from our experience it is Apparent that these methods are widely 

used, critical to system design and development work, and could be 

substantially improved: workload analysis; function allocation; task 

analysis; survey techniques; and protocol analysis. 

Workload analysis is already the subject of many ongoing research 

programs; however, it is important enough to merit expanded support: for - 

research on workload assessment methods. While the five methods named 

above are, in bUr opinion^ most deserving of research attention^ the 

order of presentation should not be construed as Indicating priorities 

among them. There is insufficient knowledge about the needs of the human 

factors coramunity to assign priorities. _ i 
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Development of New Applied Methods 

1 N 

In. discussing current and future prbbleins and trends In human factors 
appllcatlbris to system development, Melster (1980, 1982) has identified 
those informational requirements of human factors specialists that imply 
needs for the development of new applied methods. On the basis of these 
suggestions, we make general recommendations for research leading to the 
development of five new applied me;thod8l: 

1. Methods for interpreting or extrapolating task/system 
requirements into personnel requirements; 

2. Performance ieasurement methods that express measures in 
terms relative to base rates for particular system 
characteristics and/or demands; 

3. Training technology methods for translating task/abilities 
requirements into training programs; 

4. System evaluation methods--static * dynamic, and 
comparative; and 

5. Methods for describing and evaluating task or system impact 
on affective responses of personnel. 
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SUMMARY 

There Is a serious disparity between the Importance of applied 
methodologies for human factors Work, particularly systems and equipment 
design^ and the effS^ts being made to document and codify them in a 
standard manner; to educate behavioral science and engineering students 
In their use in colleges and universities; to provide coitinaing 
education in applied methods to working human factors specialists; and to 
engage in research to Improve existing applied methodologies aSd develop 
new ones. It is of great importance to document what is currently known 
about applied methods. Increasing the accessibility of information on 
existing methods would be more Valuable than developing new methods. 
What follows Is a suinmary of our recommendations with Respect to applied 
methods. 

i 

o Existing methodologies should be assessed and documented in a 
codified cbmpendiiln that provides standai-d descriptions of the most 
useful applied inethbds. This compendium would serve both as a 
comprehensive and readily available soured for learning about and as a 
basis for determining specific research needs. 

o Human factors practitioners should be surveyed to determine the. 
importarice arid frequency of use 5f existing applied methods in their 

_____ ' ' 

work; the kinds of Information most needed In huw^n factors applications 

_._ _ ' < 

for which existing applied methods are inadequate or Sbnexlstent; and 

methods for which they require descriptions and guidance for ose . 
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o Totorlals on applied tnethods should be develbp^ed to meet the 
continaing educational needs of human factors .specialists. Methods 

recdimended for the initial tutorial are: task ahalysiss; time line . 

- - - - - - - ' - - - - \ 

atTalysis; activities analysis; simulation; arid irifbrmatiori analysis. 

o Basic research should be performed to improve arid extend 
existing applifed methods. Methods in rieed of research iriclude: workload 
analysis; function allocation; task analysis; survey techniques; and 
protocol analysis. 

b Basic research is also required to devislop new methods that- can 
prbvide the J.rif bnnatibri needed by human factors specialises to do their 
wbrk. New methods needed include: (1) methods for interpreting or 
extrapolating task/system requirements into personnel selection 
requiremerits; -^!2) perfbrmarice measurement methods that express measures 
in terms relative tb base rates for particular system characteristics 
arid/br demands; (3) training technology methods for translating, 
task/abilities requirements into training programs; (4) system eV^uation 
methods— static , dynamic, and comparative; and (5) methods for describing^ 
and evaluating task or system impact on affective responses of personnel. 
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APPENDIX A 



SHORT DEFINITIONS OF APPEiED METHODS' 
REee>QffiNDED AS SUBJEeTS FOR TUTORIAL 



Task AnaiyslB 



Task an.lysl. is the proc... of analyzing functional requirement, of a 

syste. to ascertain and describe the tasks th.t people .ust perfo».. 

Task analysis has two .ajor ..p,cts: ^he fi^i.t .pacifies and des.rib.. 

the t..ks; the second and n>.r. Important a.aly..s the specified ta^ to^^ 
deten.lo. the number of p.opl. aeeded. the .ki41^ ^nd knowledge th.y 
should h.ve, and the training necessary. Reyt^ of task analysis .re 
us.d^lo the development of operating procedure, .nd technical z^inu.l. and 
the d.tennlnatlon of critical equlp.entSh.r.ct.rlstlcs and task d...nd« 
impcsed on people. The .n.lytkc .ethod Invo^. decomposition of t..k 
content into their constituent elements, such stimulus Input, required 
response, equipment output, and feedback information. 



Simulation 



V 



ERIC 



simulation IS used (1) to allow user, to e^perleSce. In advance of Its' 
operation, portions of, a sys tern that .r. more complex. .or| .dangerous, or 
sore expensive than an experiment could allow, for oS^ j2) to predict , 
performance of system, th.t do not exist. Simulation is ataman factors 

23n ^ 
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methbdblbgy only when It Is combined with bn^ of the dbiservatlonal or 
measurement methbdolbgles • And to extrapblate the observations or 
Measurements to the real world requires a determlhatloh bf the extent to 
which things that affect the observations bf Interest are realistically 
portrayed in the simulation. How to make this determi'natioh 
(cost/transfer fuhctlbn^ part versus whole task simulatibhi which things 
to simulate) is the key part bf the techhblbgy that is -gtill largely 
unresolved. In the absence bf bther effective means bf predicting the 
behavioral consequences of system design, simulation Is crucial. 



Time tine Analysis 



Time line analysiB organizes a detailed task list for the operational 

scenario and procedures into serial order and plots the times of 

_ _ _ . __j 

individual tasks in sequence against a time basie. it portrays 

sequential, parallel, repealed, and/or intermittent tasks according to 

what is done. The resulting accumulation of tasks, and total performance 

timiB can be used to a^raise: ^ 

i"^ 1. The validity of the operations to be performed in contributing 

to system objectives; > _ 

2. The feasibility of performing required tasks within the required 
timiP, > ^ 

3. Antecedent hardware and bperat ions conditions to ensure that the 
requirements of each task element are inct; . ^ >.* 
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4. Th. ccpatiblilty of di.ands,on th. operator, ensuring that 

; inticedent task. .r. Identified a«d^p.rfor.ed. re.ulr.d stills 
and perfon^ances are feasible add practical, and difficult, 
complex, or conflicting demands are avoided; and 

5. Workload d.mand., .by comparing tl:.. requirements to c.tnplete a 
task serle. to the time available for completion Within the 
constraints of a giveS systeni. - 



Information Analysis 

Info™ation an.lysis^ identifies information and its flow through a 
system, usually .s perceived from a u.^r's viewpoint. F.r Example, the 
flow of information necessary for the operation of an office differs fr, 
th^fiow of documents through that offi«. Certain syst.m actions occu: 
'to the information received, which in tur^^ecomes inputs to subsequent 
actions. Information analyse, enable hu^in factors Specialists to asse 
and design the information requirements of the user interfa 
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Activity Analysis 



. in many situations Involving field environments, simulations, or ' . 

: ■ -ck-.p., desirable^and useful to catalog the distribution aod/or • 

«-g°fnjia^ dependencies ^ workers/ activities. In activity analyiii an ' 
- • .Sser.er 'i^eriodic^lly. or .periodically; samples the work ^being performed 
, ' aSd classifies the results intd^^ set ^of categories; The data ia^ be • ^ 



ERIC ^ • 



3i 



obtained f^oi direct observation or from video or film recording.' 
individual samples are then aggregated iflto activity frequency tables or ^ 
graphs or state transition diagrams. These analyses are especially 
useful for documenting the way in which task requirements change with 
alternative system designs or environments or for estimates of relative 
cost effectiveness, manning requirements, or simply for understanding how 
Individuals or groups spend thcflr time. 
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